Technology Category
Machine Learning: Probabilistic Circuits and Graphical ModelsReasoning under Uncertainty: Probabilistic ProgrammingKnowledge Representation and Reasoning: Computational Complexity of Reasoning
On the Hardness of Approximating Distributions with Probabilistic Circuits
🤖 AI Summary
This paper investigates the feasibility and complexity of approximating arbitrary probability distributions using probabilistic circuits (PCs) under small approximation errors, aiming to mitigate exponential size blowup caused by structural constraints such as decomposability and determinism.
Method: Leveraging tools from computational complexity theory, *f*-divergence analysis, and polynomial-time reductions, the authors establish tight theoretical lower bounds on approximate representational complexity.
Contribution/Results: The work delivers two key results: (i) it proves that achieving ε-approximation—in the sense of *f*-divergence—of any model with efficiently computable marginal distributions is NP-hard; and (ii) it reveals an exponential gap in approximation size between decomposable and deterministic PCs. These findings rigorously quantify the fundamental trade-off between structural constraints and approximation efficiency, providing an unattainable complexity benchmark for PC architecture design.
Technology Category
Application Category
📝 Abstract
A fundamental challenge in probabilistic modeling is balancing expressivity and tractable inference. Probabilistic circuits (PCs) aim to directly address this tradeoff by imposing structural constraints that guarantee efficient inference of certain queries while maintaining expressivity. Since inference complexity on PCs depends on circuit size, understanding the size bounds across circuit families is key to characterizing the tradeoff between tractability and expressive efficiency. However, expressive efficiency is often studied through exact representations, where exactly encoding distributions while enforcing various structural properties often incurs exponential size blow-ups. Thus, we pose the following question: can we avoid such size blow-ups by allowing some small approximation error? We first show that approximating an arbitrary distribution with bounded $f$-divergence is $mathsf{NP}$-hard for any model that can tractably compute marginals. We then prove an exponential size gap for approximation between the class of decomposable PCs and additionally deterministic PCs.
Problem
Research questions and friction points this paper is trying to address.
Balancing expressivity and tractable inference in probabilistic modeling
Understanding size bounds in probabilistic circuits for efficient inference
Investigating approximation to avoid exponential size blow-ups in distributions
Innovation
Methods, ideas, or system contributions that make the work stand out.
Balancing expressivity and tractable inference
Approximating distributions with bounded divergence
Exponential size gap for approximation
🔎 Similar Papers
No similar papers found.
