Skew-symmetric approximations of posterior distributions

2024-09-21

AI Summary

Bayesian posteriors are often skewed, whereas mainstream deterministic approximations—such as Laplace’s method and variational Bayes—rely on symmetric densities (e.g., Gaussians), leading to systematic bias and reduced accuracy. Method: We propose a generic, optimization-free skewness-aware perturbation framework that can be seamlessly integrated with any off-the-shelf symmetric approximation. Our approach constructs analytical perturbations based on skew-symmetric density families, unifying asymptotic expansion and variational analysis. Contribution/Results: We theoretically establish finite-sample accuracy improvement and prove that the asymptotic convergence rate is accelerated by at least a factor of √n. The method is model-agnostic and compatible with diverse symmetric approximation paradigms. Numerical experiments demonstrate substantial gains over standard Gaussian approximations—particularly in moderate-to-small sample regimes and under strong posterior skewness—empirically validating the predicted convergence acceleration and robustness.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Routinely-implemented deterministic approximations of posterior distributions from, e.g., Laplace method, variational Bayes and expectation-propagation, generally rely on symmetric approximating densities, often taken to be Gaussian. This choice facilitates optimization and inference, but typically affects the quality of the overall approximation. In fact, even in basic parametric models, the posterior distribution often displays asymmetries that yield bias and reduced accuracy when considering symmetric approximations. Recent research has moved towards more flexible approximating densities that incorporate skewness. However, current solutions are model-specific, lack general supporting theory, increase the computational complexity of the optimization problem, and do not provide a broadly-applicable solution to include skewness in any symmetric approximation. This article addresses such a gap by introducing a general and provably-optimal strategy to perturb any off-the-shelf symmetric approximation of a generic posterior distribution. Crucially, this novel perturbation is derived without additional optimization steps, and yields a similarly-tractable approximation within the class of skew-symmetric densities that provably enhances the finite-sample accuracy of the original symmetric approximation, and, under suitable assumptions, improves its convergence rate to the exact posterior by at least a $sqrt{n}$ factor, in asymptotic regimes. These advancements are illustrated in numerical studies focusing on skewed perturbations of state-of-the-art Gaussian approximations.

Problem

• Address bias in symmetric posterior approximations
• Develop general skew-symmetric perturbation method
• Improve accuracy without additional optimization steps

Innovation

• Introduces skew-symmetric perturbations for symmetric approximations
• No additional optimization steps required
• Improves convergence rate by √n factor