Theoretical guarantees for stochastic gradient sampling methods via Gaussian convolution inequalities

2026-04-27

AI Summary

This work addresses the lack of sharp non-asymptotic theoretical guarantees for the invariant measure bias in stochastic gradient Markov Chain Monte Carlo (SG-MCMC) methods. Under mild assumptions, it investigates the accuracy of the invariant measure for stochastic gradient kinetic Langevin dynamics. By introducing a novel Gaussian convolution inequality, the paper establishes, for the first time, a tight first-order Wasserstein distance bound that scales linearly with the step size. This result resolves a long-standing open problem in the field, significantly strengthening the theoretical foundations of SG-MCMC algorithms. Moreover, it provides a quantitative characterization of the invariant measure bias, and the newly developed inequality is of independent interest with potential applications beyond this context.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Learning with Manifolds
• Search and Optimization: Non-convex Optimization

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Web measurements

Abstract

We derive first-order (in the stepsize) bounds on the bias in Wasserstein distances of the invariant measure of stochastic gradient kinetic Langevin dynamics with minimal assumptions on the stochastic gradient noise. These bounds sharpen existing non-asymptotic guarantees for stochastic-gradient MCMC methods and provide a quantitative resolution of a previously open problem on invariant measure accuracy. The main technical ingredients are new Gaussian convolution inequalities controlling the Wasserstein-$p$ distance between a Gaussian convolved with a mean-zero perturbation and the Gaussian itself. We anticipate that these inequalities will be of independent interest beyond the present application.

Problem

• stochastic gradient sampling
• invariant measure accuracy
• Wasserstein distance
• non-asymptotic guarantees
• Gaussian convolution inequalities

Innovation

• Gaussian convolution inequalities
• Wasserstein distance
• stochastic gradient MCMC
• invariant measure bias
• kinetic Langevin dynamics