adaptive belief propagation

Conventional variational approximations (e.g., Bethe, tree-reweighted) for highly coupled probabilistic graphical models fail under parameter perturbations or entropy approximation mismatches. Method: This paper proposes an adaptive free energy approximation framework that systematically characterizes the dual influence of parameter variations and entropy approximation errors on free energy construction. Based on this analysis, we design a model-driven, dynamically adjustable free energy functional capable of online adaptation to both structural and parametric changes. The approach integrates variational inference, convex analysis, and adaptive parametric modeling to ensure theoretical consistency and computational tractability. Contribution/Results: Experiments across diverse intractable graphical models demonstrate that our method significantly improves accuracy in marginal distribution and partition function estimation, achieves more stable convergence, and exhibits superior generalization compared to state-of-the-art convex and non-convex approximation methods.

This work addresses the degradation of belief propagation (BP) accuracy on loopy networks. We propose embedding non-iterative belief (NIB) message passing into the generalized belief propagation framework to systematically improve accuracy in two fundamental tasks: percolation threshold prediction and sparse matrix spectral estimation. Unlike classical Kikuchi cluster-based methods (e.g., KCN), NIB avoids loop-induced cyclic dependencies, thereby enhancing inference stability and convergence—particularly on high-loop-density graphs. Experiments on Erdős–Rényi (ER), Barabási–Albert (BA), and real-world networks demonstrate over 30% reduction in percolation critical-point estimation error. Moreover, NIB achieves superior reconstruction of sparse matrix eigenvalue distributions, especially near spectral edges. This study extends the applicability of message-passing algorithms to interdisciplinary problems at the interface of statistical physics and spectral graph theory, establishing a new paradigm for efficient approximate inference on complex networks.

Belief propagation (BP) struggles to accurately estimate order parameters and susceptibilities in finite-size sparse networks due to persistent global symmetry, especially near phase transitions. Method: We propose a symmetry-breaking strategy that fixes the state of a high-degree “source” node—explicitly breaking global symmetry without increasing computational complexity. This approach is applicable to tree-like and sparse graphs with few cycles. Contribution/Results: By integrating symmetry-breaking mechanisms from percolation and Ising models, our method yields more robust message passing and significantly improves BP’s accuracy in estimating order parameters and susceptibilities near critical points. Experiments across diverse sparse network topologies demonstrate consistent performance gains, establishing a new paradigm for statistical inference in finite systems. The method preserves BP’s scalability while effectively capturing finite-size effects, offering a principled and computationally efficient enhancement to standard BP for inference on sparse graphical models.

This paper addresses the computational intractability of directly minimizing Expected Free Energy (EFE) under cognitive uncertainty. To resolve this, we propose a variational inference framework grounded in factor graph message passing. Our core method reformulates EFE minimization as variational free energy optimization with explicit cognitive priors; leveraging factor graph modeling and state-space decomposition, it transforms combinatorial policy search into scalable, distributed message-passing inference. This constitutes a substantive bridge from active inference theory to executable algorithms. Empirical evaluation on stochastic grid-worlds and partially observable MiniGrid tasks demonstrates that agents employing our approach exhibit markedly more robust path planning and systematic information-seeking behavior—outperforming baseline methods such as KL control by significant margins.

This work establishes the theoretical foundations of distributed (non-)Bayesian inference within the frequentist framework, addressing statistical reliability in multi-agent collaborative inference over decentralized networks. Method: Integrating random graph theory, asymptotic statistics, and distributed optimization, the study rigorously analyzes posterior consistency, asymptotic normality, and posterior contraction rates under general network topologies. Contribution/Results: It provides the first formal characterization of how communication graph connectivity governs the statistical–communication efficiency trade-off. Theoretically, it proves that—under mild connectivity conditions—distributed inference preserves parametric efficiency while enhancing robustness in uncertainty quantification; it further derives explicit, network-size-dependent bounds on posterior contraction rates. The framework accommodates time-varying topologies and canonical models including exponential families, logistic regression, and decentralized detection, delivering verifiable frequentist guarantees for large-scale distributed learning.

This study addresses the lack of theoretical guarantees for belief propagation (BP) in sparse, loopy factor graphs under non-Gaussian settings. By leveraging the central limit theorem, the authors analyze the statistical properties of BP message passing and prove that, under four reasonable assumptions, the marginal beliefs over variables converge to a Gaussian distribution as iterations proceed. This work provides the first theoretical convergence guarantee for Gaussian belief propagation (GBP) in non-Gaussian, sparse graphical models, uncovering an intrinsic “Gaussianization” mechanism inherent to BP. Experimental validation on stereo vision depth estimation demonstrates that variable beliefs become markedly Gaussian after only a few iterations, thereby substantiating the empirical success and broad applicability of GBP in spatial AI and related domains.

This work addresses the computational intractability of probabilistic inference in high-dimensional Bayesian networks, which stems from the exponential complexity of their joint distributions. The authors propose a novel framework based on directed convex subgraph decomposition, introducing a minimal d-decomposition tree as an alternative to conventional junction trees. This structure decomposes the joint distribution into low-dimensional submodels that can be learned and stored independently. The approach inherently supports localized and parallelized inference, achieving substantial gains in computational efficiency while preserving inference accuracy—particularly advantageous for low-dimensional queries. The core contributions lie in an improved structural decomposition mechanism and highly efficient algorithms for parallel parameter estimation and inference.

This paper studies Bayesian inference in the spiked Wigner model—recovering a planted Boolean spike from a noisy symmetric matrix. To bridge the unclear connection between Markov chain Monte Carlo (MCMC) and approximate message passing (AMP), we propose the Restricted Gaussian Dynamics (RGD) auxiliary chain, which rigorously establishes, for the first time, a one-dimensional recursive equivalence between Glauber dynamics on the annealed posterior and AMP iterations. We prove that RGD converges rapidly from a warm start to the correlation-space fixed point, achieving Bayes-optimal recovery. Moreover, under the Sherrington–Kirkpatrick (SK) model’s mixture assumption, RGD reproduces the nontrivial inference phase transition threshold. This work provides the first dynamical-level unifying framework linking MCMC and variational inference, offering rigorous insights into the interplay between sampling-based and iterative deterministic algorithms in high-dimensional Bayesian estimation.

This work addresses the issue that expectation propagation (EP) may generate non-integrable beliefs during iteration, leading to inference failure—particularly in Bayesian inference tasks such as generalized linear models. To resolve this, the authors propose two novel EP frameworks that enforce integrability of beliefs by constraining the belief space itself rather than imposing restrictive conditions on messages, thereby preserving message flexibility while rigorously guaranteeing belief integrability. These frameworks are derived from a constrained Bethe free energy optimization perspective, seamlessly integrating variational inference with message-passing mechanisms to yield stable iterative update rules. Experimental results demonstrate that the proposed methods effectively avoid non-integrable beliefs in signal recovery tasks under generalized linear models, significantly improving estimation accuracy and extending the applicability of EP to non-standard probabilistic models.

This work addresses the computational redundancy inherent in Markov chain Monte Carlo (MCMC) inference for probabilistic programming by proposing an incremental inference method based on dynamic dependency graphs. The approach compiles probabilistic programs into reactive computation graphs, enabling selective recomputation of only those subgraphs affected by updates to random variables, thereby substantially reducing the per-iteration sampling cost. Notably, this study is the first to integrate functional reactive programming with probabilistic programming, unifying the representational frameworks of Bayesian networks—formalized as applicative functors—and general-purpose probabilistic programs expressed as monads. The method achieves significant improvements in MCMC efficiency while preserving inference accuracy.