THOI: An efficient and accessible library for computing higher-order interactions enhanced by batch-processing

2025-01-06

AI Summary

Quantifying higher-order interactions (HOIs) in continuous systems faces fundamental challenges—including difficulty in estimating joint entropy, computational intractability due to combinatorial explosion, and poor scalability. Method: We propose the first HOI analysis framework integrating Gaussian copula-based entropy estimation with tensorized batch processing and multi-backend parallel computation (CPU/GPU/TPU). Leveraging copula modeling and PyTorch tensor operations, our approach incorporates scalable optimization strategies to mitigate the curse of dimensionality. Contribution/Results: Our method enables the first information-theoretic decomposition (e.g., redundancy, synergy) of HOIs on real-world datasets with >1,000 variables—achieving >10× speedup over state-of-the-art tools. It successfully identifies consciousness-state-specific higher-order neural synergies in fMRI data. Furthermore, we conduct systematic benchmarking across 900+ real and synthetic datasets, releasing an efficient, open-source, and fully reproducible Python toolkit for studying nonlinear dynamics in complex systems.

Technology Category

Cognitive Modeling & Cognitive Systems: Neural Spike CodingMachine Learning: Information TheoryHumans and AI: Brain-Sensing and Analysis

Application Category

Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphsEconomics, Online Markets and Human Computation: Cost models of using LLMs in production systemsSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Complex systems are characterized by nonlinear dynamics, multi-level interactions, and emergent collective behaviors. Traditional analyses that focus solely on pairwise interactions often oversimplify these systems, neglecting the higher-order interactions critical for understanding their full collective dynamics. Recent advances in multivariate information theory provide a principled framework for quantifying these higher-order interactions, capturing key properties such as redundancy, synergy, shared randomness, and collective constraints. However, two major challenges persist: accurately estimating joint entropies and addressing the combinatorial explosion of interacting terms. To overcome these challenges, we introduce THOI (Torch-based High-Order Interactions), a novel, accessible, and efficient Python library for computing high-order interactions in continuous-valued systems. THOI leverages the well-established Gaussian copula method for joint entropy estimation, combined with state-of-the-art batch and parallel processing techniques to optimize performance across CPU, GPU, and TPU environments. Our results demonstrate that THOI significantly outperforms existing tools in terms of speed and scalability. For larger systems, where exhaustive analysis is computationally impractical, THOI integrates optimization strategies that make higher-order interaction analysis feasible. We validate THOI accuracy using synthetic datasets with parametrically controlled interactions and further illustrate its utility by analyzing fMRI data from human subjects in wakeful resting states and under deep anesthesia. Finally, we analyzed over 900 real-world and synthetic datasets, establishing a comprehensive framework for applying higher-order interaction (HOI) analysis in complex systems.

Problem

• High-order interaction
• Complex system analysis
• Entropy computation

Innovation

• High-order Interaction
• Gaussian Copula Method
• Parallel Computing Technology