Rates of Convergence of Generalised Variational Inference Posteriors under Prior Misspecification

2025-10-03

AI Summary

This paper addresses the convergence rate and robustness of generalized variational inference (GVI) under prior misspecification, particularly confronting challenges arising from non-KL divergences, restricted measure spaces, and intractable posteriors. We establish sufficient conditions for the existence and uniqueness of the GVI posterior on Polish spaces, prove its almost-sure convergence to the loss-minimizing neighborhood, and derive a prior-agnostic upper bound on the convergence rate. Innovatively, we introduce bounded-divergence constraints and measure-space truncation, integrating functional analysis and probability measure theory to provide the first unified convergence guarantees for both centralized and federated GVI under severe prior misspecification. Our results significantly enhance robustness to model misspecification and lay a rigorous theoretical foundation for distributed learning and other complex inference settings.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Probabilistic Inference
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Security and Privacy: Large-scale security measurements

Abstract

We prove rates of convergence and robustness to prior misspecification within a Generalised Variational Inference (GVI) framework with bounded divergences. This addresses a significant open challenge for GVI and Federated GVI that employ a different divergence to the Kullback--Leibler under prior misspecification, operate within a subset of possible probability measures, and result in intractable posteriors. Our theoretical contributions cover severe prior misspecification while relying on our ability to restrict the space of possible GVI posterior measures, and infer properties based on this space. In particular, we are able to establish sufficient conditions for existence and uniqueness of GVI posteriors on arbitrary Polish spaces, prove that the GVI posterior measure concentrates on a neighbourhood of loss minimisers, and extend this to rates of convergence regardless of the prior measure.

Problem

• Establishes convergence rates for variational inference with bounded divergences
• Addresses robustness to prior misspecification in generalized variational inference
• Proves posterior concentration on neighborhoods of loss minimizers

Innovation

• Proves convergence rates for bounded divergence GVI
• Establishes GVI posterior existence on Polish spaces
• Shows posterior concentration near loss minimizers