Stability of Mean-Field Variational Inference

2025-06-09

AI Summary

This paper investigates the stability of mean-field variational inference (MFVI) solutions under strongly log-concave target distributions. Addressing MFVI’s inherent nonconvexity and dimension dependence, we introduce— for the first time—a novel analytical framework based on linearized optimal transport, which recasts the original problem as a convex transport-map optimization. This reformulation enables us to establish dimension-free Lipschitz continuity of MFVI solutions with respect to the target distribution and to prove their Fréchet differentiability with respect to the potential function, with the derivative characterized by an elliptic PDE. Explicit stability bounds are derived under the 2-Wasserstein metric. Our theoretical advances provide the first quantitative stability guarantees and differentiability foundation for robust Bayesian inference and distributed stochastic control.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Learning with Manifolds
• Search and Optimization: Non-convex Optimization

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Mean-field variational inference (MFVI) is a widely used method for approximating high-dimensional probability distributions by product measures. This paper studies the stability properties of the mean-field approximation when the target distribution varies within the class of strongly log-concave measures. We establish dimension-free Lipschitz continuity of the MFVI optimizer with respect to the target distribution, measured in the 2-Wasserstein distance, with Lipschitz constant inversely proportional to the log-concavity parameter. Under additional regularity conditions, we further show that the MFVI optimizer depends differentiably on the target potential and characterize the derivative by a partial differential equation. Methodologically, we follow a novel approach to MFVI via linearized optimal transport: the non-convex MFVI problem is lifted to a convex optimization over transport maps with a fixed base measure, enabling the use of calculus of variations and functional analysis. We discuss several applications of our results to robust Bayesian inference and empirical Bayes, including a quantitative Bernstein--von Mises theorem for MFVI, as well as to distributed stochastic control.

Problem

• Studying stability of mean-field variational inference under distribution changes
• Establishing Lipschitz continuity of MFVI optimizer in 2-Wasserstein distance
• Analyzing differentiability of MFVI optimizer with respect to target potential

Innovation

• Uses linearized optimal transport approach
• Lifts MFVI to convex optimization problem
• Applies calculus of variations techniques