apply expectation propagation

Expectation Propagation (EP) suffers from numerical instability and convergence failure under Gaussian projection due to the emergence of negative-variance messages, which cause divergent integrals. Method: Within the linear model framework, this work first systematically characterizes the structural dependencies among EP messages; based on this analysis, it proposes two proactive mitigation strategies—“non-persistent” and “persistent”—that intrinsically prevent the generation of ill-posed, infinitely valued messages. Unlike heuristic regularization approaches, our method relies solely on message propagation path analysis and introduces no additional hyperparameters or approximate corrections. Contribution/Results: Experiments demonstrate that the proposed strategies substantially improve EP’s robustness and convergence speed, effectively suppressing negative-variance messages in both Bayesian linear regression and generalized linear models. This work provides both theoretical insight into EP’s instability mechanisms and a practical, parameter-free tool for stable EP deployment.

This work addresses the issue that expectation propagation (EP) may generate non-integrable beliefs during iteration, leading to inference failure—particularly in Bayesian inference tasks such as generalized linear models. To resolve this, the authors propose two novel EP frameworks that enforce integrability of beliefs by constraining the belief space itself rather than imposing restrictive conditions on messages, thereby preserving message flexibility while rigorously guaranteeing belief integrability. These frameworks are derived from a constrained Bethe free energy optimization perspective, seamlessly integrating variational inference with message-passing mechanisms to yield stable iterative update rules. Experimental results demonstrate that the proposed methods effectively avoid non-integrable beliefs in signal recovery tasks under generalized linear models, significantly improving estimation accuracy and extending the applicability of EP to non-standard probabilistic models.

This work addresses the intractability of exact Bayesian inference for latent states in continuous-time Markov jump processes—such as chemical reaction networks and stochastic Lotka–Volterra systems. We propose an analytically tractable expectation propagation (EP) framework grounded in entropy matching: by embedding entropy matching into the EP formalism, we derive closed-form approximations to the latent-state posterior distribution and integrate them with an approximate EM algorithm for efficient parameter estimation. Our approach overcomes the high computational complexity and poor scalability inherent in conventional methods for discrete-state continuous-time models. Evaluated on multiple systems biology benchmarks, the method achieves high-accuracy latent-state inference and parameter estimation while significantly improving computational scalability and practical applicability.

To address the challenge of scarce clean training data in Bayesian inverse problems, this paper proposes the first framework for learning theoretically grounded diffusion priors solely from incomplete and noisy observations. Methodologically, we embed diffusion probabilistic modeling into an Expectation-Maximization (EM) algorithm: the E-step estimates the latent variable posterior via iterative denoising sampling, while the M-step updates diffusion model parameters by maximizing the marginal likelihood. Our key contributions are: (1) the first provably consistent learning of diffusion priors directly from noisy and/or missing observations; and (2) an unconditional posterior sampling strategy that eliminates reliance on assumptions about the forward process. Experiments demonstrate that the learned prior achieves performance on par with fully supervised models in downstream inverse tasks—including denoising and inpainting—while ensuring rigorous generative consistency and theoretical guarantees.

This work establishes the theoretical foundations of distributed (non-)Bayesian inference within the frequentist framework, addressing statistical reliability in multi-agent collaborative inference over decentralized networks. Method: Integrating random graph theory, asymptotic statistics, and distributed optimization, the study rigorously analyzes posterior consistency, asymptotic normality, and posterior contraction rates under general network topologies. Contribution/Results: It provides the first formal characterization of how communication graph connectivity governs the statistical–communication efficiency trade-off. Theoretically, it proves that—under mild connectivity conditions—distributed inference preserves parametric efficiency while enhancing robustness in uncertainty quantification; it further derives explicit, network-size-dependent bounds on posterior contraction rates. The framework accommodates time-varying topologies and canonical models including exponential families, logistic regression, and decentralized detection, delivering verifiable frequentist guarantees for large-scale distributed learning.

This work addresses the exponential growth in the number of components that arises when multiplying Gaussian mixture models (GMMs), which hinders efficient approximation of univariate factorized distributions. To overcome this challenge, the authors propose a multivariate modeling approach based on variable duplication, reformulating the original problem into a graphical model amenable to local computation. Efficient inference is achieved by integrating Gaussian belief propagation (GaBP) with expectation propagation (EP). Furthermore, an improved non-integrable belief handling strategy is introduced to enhance the stability and applicability of the approximation. The resulting method enables scalable and stable approximation of univariate distributions involving multiple GMM factors, effectively circumventing component explosion and significantly improving computational efficiency in joint estimation and detection tasks within communication systems.

This work addresses the challenge of surrogate-induced overconfidence in Bayesian inverse problems, where computationally expensive forward models are approximated by surrogates whose uncertainty is often neglected. The authors propose the Expected Posterior (EP) as a principled benchmark for propagating surrogate uncertainty, deriving it for the first time from decision-theoretic and modular Bayesian inference principles. They demonstrate that the commonly used heuristic Expected Utility Posterior (EUP) incurs systematic bias when surrogate uncertainty is non-uniform. To enable practical computation of EP, they develop a randomized kernel-preconditioned Crank–Nicolson (RKpCN) MCMC algorithm, which efficiently approximates the EP even with infinite-dimensional Gaussian process surrogates. This approach significantly enhances the reliability of posterior inference in high-dimensional settings.

This study addresses the challenges of iterative scheduling, negative precision, and Dirac factor collapse inherent in Expectation Propagation (EP) and Variational Message Passing (VMP) within approximate message passing on factor graphs. To this end, we propose a direct message approximation framework that constructs messages via consistency conditions of normalizable factors, thereby eliminating inner loops and learning rate hyperparameters to enable efficient inference through a single forward-backward pass. Theoretically, we establish a main theorem bounding the KL divergence and provide the first O(1/r²) error guarantee for backward messages at product factors. Experimental results demonstrate that the proposed method effectively overcomes the negative precision issue while appropriately inflating predictive uncertainty in data-sparse regions.

This work addresses the challenging setting of Bayesian inference where the posterior distribution is highly non-Gaussian, gradients are unavailable, and model evaluations are computationally expensive. To tackle this, the authors propose an iterative variational framework that integrates geometric annealing, multi-fidelity modeling, and gradient-free measure transport. By constructing a measure transport surrogate that reuses costly high-fidelity simulations and incorporating importance-weighted multi-set quadrature rules, the method enables efficient posterior sampling and accurate expectation estimation. Demonstrated on low-dimensional but strongly non-Gaussian inverse problems governed by partial differential equations, the approach substantially improves posterior approximation accuracy, yields high-quality independent samples, and achieves a favorable balance between computational efficiency and statistical fidelity.

This work addresses the challenge in black-box variational inference (BBVI) where stochastic gradients exhibit unbounded variance and only satisfy the relatively weak Blum–Gladyshev (BG) condition, rendering conventional optimization theory inapplicable. Focusing on the elliptical location-scale family of distributions, the authors propose a minibatch projected stochastic gradient descent method that integrates preconditioning with dynamic batching. Under the BG condition, they establish, for the first time, the rigorous existence of an evidence lower bound (ELBO) maximizer and provide both finite-time and asymptotic convergence guarantees for the algorithm. Theoretical analysis and empirical experiments demonstrate that the proposed approach significantly enhances the stability and effectiveness of BBVI in settings with unbounded gradient variance.