🤖 AI Summary
This paper addresses the challenge of uniformly detecting higher-order feature interactions—specifically synergistic, redundant, and independent relationships—which are difficult to characterize with existing methods. We propose a geometric analysis framework based on random sequential feature addition. By modeling how individual feature contributions evolve as features are added in varying orders, we observe that their trajectories in a two-dimensional plane consistently exhibit an L-shaped pattern. Leveraging this geometric regularity, we define the continuous, interpretable L-score (ranging from −1 to +1), which enables inference of third- and higher-order interaction structures using only pairwise interaction measurements. Our method imposes no assumptions about underlying model architecture, requires no gradient computation or predefined distance metrics, and yields unbiased identification of feature dominance and interaction type. Extensive experiments across diverse domains demonstrate accurate discrimination among synergistic (e.g., Y = X₁X₂), redundant (e.g., X₁ ≈ X₂), and independent relationships—achieving, for the first time, model-agnostic, quantitative, and interpretable decomposition of higher-order interactions.
📝 Abstract
Many systems exhibit complex interactions between their components: some features or actions amplify each other's effects, others provide redundant information, and some contribute independently. We present a simple geometric method for discovering interactions and redundancies: when elements are added in random sequential orders and their contributions plotted over many trials, characteristic L-shaped patterns emerge that directly reflect interaction structure. The approach quantifies how the contribution of each element depends on those added before it, revealing patterns that distinguish interaction, independence, and redundancy on a unified scale. When pairwise contributions are visualized as two--dimensional point clouds, redundant pairs form L--shaped patterns where only the first-added element contributes, while synergistic pairs form L--shaped patterns where only elements contribute together. Independent elements show order--invariant distributions. We formalize this with the L--score, a continuous measure ranging from $-1$ (perfect synergy, e.g. $Y=X_1X_2$) to $0$ (independence) to $+1$ (perfect redundancy, $X_1 approx X_2$). The relative scaling of the L--shaped arms reveals feature dominance in which element consistently provides more information. Although computed only from pairwise measurements, higher--order interactions among three or more elements emerge naturally through consistent cross--pair relationships (e.g. AB, AC, BC). The method is metric--agnostic and broadly applicable to any domain where performance can be evaluated incrementally over non-repeating element sequences, providing a unified geometric approach to uncovering interaction structure.