🤖 AI Summary
This work addresses the systematic underestimation of posterior functionals—such as expectations and variances—in structured mean-field variational inference, which arises from neglected cross-block dependencies whose geometric origin remains unclear. We introduce a differential-geometric framework based on the tangent space induced by the variational family and establish, for the first time, a precise link between functional bias and tangent space geometry. Specifically, we show that the component orthogonal to the tangent space dominates the first-order bias, whereas the component within the tangent space—spanned by block-additive functions—contributes only to second-order bias. By integrating local asymptotic normality with functional asymptotic expansions, we derive an explicit expression for the bias, offering a geometric explanation for the distortion of cross-block dependencies inherent in mean-field approximations.
📝 Abstract
Variational inference approximates Bayesian posterior distributions by projecting onto a tractable family of distributions. While most theoretical analyses evaluate the quality of this approximation using global divergence measures, many applications rely on specific posterior summaries such as expectations, variances, or tail probabilities. We develop a geometric framework for analyzing the bias of posterior functionals under variational approximations. We show that the leading-order bias of a posterior functional is determined by its component orthogonal to the variational tangent space induced by the variational family. Functionals aligned with this space incur only second-order bias. For structured mean-field variational families we characterize the tangent space explicitly and show that it consists of block-additive functions of the parameter blocks, while interaction components determine the leading-order bias. Under standard local asymptotic normality conditions we further derive explicit asymptotic expansions for the bias of posterior functionals and show that omitted interaction directions produce first-order distortion of cross-block dependence measures. These results provide a geometric explanation for several well-known properties of mean-field variational inference, including the systematic distortion of cross-block dependencies.