Gaussian Invariant Markov Chain Monte Carlo

2025-06-26

AI Summary

To address the low sampling efficiency of Markov chain Monte Carlo (MCMC) methods in high-dimensional latent Gaussian models, this paper proposes a Gaussian-invariant MCMC framework that unifies Gaussian-invariant variants of the random-walk Metropolis (RWM), Metropolis–Adjusted Langevin Algorithm (MALA), and Hessian/Manifold MALA. It introduces, for the first time, control variates derived from analytical solutions to the Poisson equation under Gaussian targets, substantially reducing estimator variance even when the target distribution is intractable. Theoretically, the work quantifies the relationship between Gaussianity and key MCMC properties—including optimal acceptance rate, scaling parameter, and geometric ergodicity—and establishes a rigorous optimal scaling theory with adaptive optimality guarantees. Empirical evaluations demonstrate that the proposed method significantly outperforms state-of-the-art samplers on high-dimensional Gaussian models, achieving variance reductions of several orders of magnitude, while maintaining both theoretical rigor and practical efficacy.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

We develop sampling methods, which consist of Gaussian invariant versions of random walk Metropolis (RWM), Metropolis adjusted Langevin algorithm (MALA) and second order Hessian or Manifold MALA. Unlike standard RWM and MALA we show that Gaussian invariant sampling can lead to ergodic estimators with improved statistical efficiency. This is due to a remarkable property of Gaussian invariance that allows us to obtain exact analytical solutions to the Poisson equation for Gaussian targets. These solutions can be used to construct efficient and easy to use control variates for variance reduction of estimators under any intractable target. We demonstrate the new samplers and estimators in several examples, including high dimensional targets in latent Gaussian models where we compare against several advanced methods and obtain state-of-the-art results. We also provide theoretical results regarding geometric ergodicity, and an optimal scaling analysis that shows the dependence of the optimal acceptance rate on the Gaussianity of the target.

Problem

• Develop Gaussian invariant MCMC sampling methods
• Improve ergodic estimators' statistical efficiency
• Construct efficient control variates for variance reduction

Innovation

• Gaussian invariant versions of RWM, MALA, and Hessian MALA
• Exact analytical solutions for Gaussian targets
• Efficient control variates for variance reduction