Modeling Discrete Data with High-Order Vector Potts Models

2026-06-02

AI Summary

This work addresses the challenge of modeling high-order (third-order and beyond) correlations in discrete high-dimensional data by introducing a class of q-state spin maximum entropy models that incorporate arbitrary high-order and long-range interactions, generalizing binary high-order minimal complexity models to the general discrete setting. Leveraging discrete Fourier analysis and gauge transformations, the study reveals a unified structural equivalence among interaction models of different orders under gauge symmetry. Furthermore, by deriving a closed-form expression for marginal likelihood via a loop expansion of the partition function, the method enables efficient model selection. Empirical validation on multiple real-world datasets demonstrates both the effectiveness of the approach in capturing high-order dependencies and its computational advantages.

Abstract

Modeling high-dimensional data is challenging, yet essential to understanding many complex systems. Maximum entropy models such as Ising and Potts models have been used extensively to capture pairwise interactions from correlation patterns in data, allowing to infer graphical representations of complex systems from observations (e.g., from protein sequences or neural population activity). Recently, there has been growing interest in modeling higher-order correlation patterns involving simultaneously three or more variables. While progress has been made in binary data with high-order Ising models, we extend this framework to the more general case of discrete data. We introduce q-state spin models, a complete family of maximum entropy models that generalize the vector Potts model to include long-range and arbitrary high-order interactions. In the pairwise case, our models allow for more diverse interaction types compared to the standard vector Potts model. We discuss their statistical interpretation with examples and relate them to discrete Fourier analysis. Using a loop expansion of the partition function, we show that the statistical properties of spin models are fully captured by the algebraic structure of their interactions. We define gauge transformations under which this structure, and thus the partition function, remains invariant. Models equivalent under gauge transformations can be seen as different representations of the same abstract statistical model, despite generally having interactions of different orders, extending results from the binary case. For practical application to data analysis, we focus on a subset of models known in the binary case as Minimally Complex Models, generalizing them to discrete data. We obtain a closed-form expression for the marginal likelihood of these models, enabling fast model selection. We illustrate their use with simple real-world examples.

Problem

• discrete data
• high-order interactions
• Potts models
• maximum entropy models
• correlation patterns

Innovation

• high-order interactions
• q-state spin models
• maximum entropy models
• gauge invariance
• discrete data modeling