Markov Combinations of Discrete Statistical Models

2025-09-23

AI Summary

Markov combinations—well-established for continuous models—have remained undefined for discrete statistical models, particularly those involving categorical variables. Method: This paper introduces the first systematic generalization of Markov combinations to the discrete domain. It proposes a novel marginal fusion operation ensuring that the marginal distribution of the combined model strictly contains those of the constituent models. Multiple variants are developed to characterize sampling mechanisms, model invariances, and closure properties over regular exponential families, staged tree models, and discrete copulas. Leveraging probabilistic graphical models and statistical manifold theory, we design a computationally tractable parameter estimation framework supporting maximum likelihood inference. Results: We prove theoretical closure and consistency of the marginal fusion operation across key discrete model classes. This establishes a new paradigm for modeling and inference with complex categorical data, enabling principled compositionality while preserving interpretability and statistical guarantees.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization
• Machine Learning: Probabilistic Circuits and Graphical Models

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Markov combination is an operation that takes two statistical models and produces a third whose marginal distributions include those of the original models. Building upon and extending existing work in the Gaussian case, we develop Markov combinations for categorical variables and their statistical models. We present several variants of this operation, both algorithmically and from a sampling perspective, and discuss relevant examples and theoretical properties. We describe Markov combinations for special models such as regular exponential families, discrete copulas, and staged trees. Finally, we offer results about model invariance and the maximum likelihood estimation of Markov combinations.

Problem

• Develops Markov combinations for categorical variables and models
• Presents algorithmic and sampling variants of the operation
• Describes combinations for exponential families, copulas, and staged trees

Innovation

• Markov combinations for categorical variables
• Algorithmic and sampling variants presented
• Model invariance and maximum likelihood results