Sample continuation in Bayesian hierarchical model via variational inference

2026-04-16

AI Summary

This work addresses the challenge that posterior distributions in ill-posed Bayesian inverse problems are highly sensitive to prior hyperparameters and often analytically intractable. To overcome this, the authors propose a Stein variational gradient descent (SVGD) framework that integrates Birth-Death sampling with adaptive kernel bandwidth optimization. By tracking the evolution of posterior samples along continuous paths of varying prior hyperparameters, the method effectively navigates across separated modes in multimodal posteriors, enabling stable continuation from unimodal to multimodal regimes. Experiments on non-conjugate sparsity-promoting hierarchical models demonstrate that the proposed approach robustly and efficiently characterizes posterior sensitivity to changes in prior assumptions, offering a novel computational tool for stability analysis in Bayesian inference.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Posterior distributions arising in ill-posed Bayesian inverse problems are often both analytically intractable and highly sensitive to parameters of the chosen prior family. We aim to understand the sensitivity of intractable posterior distributions to changes in prior assumptions by tracking how a sample representation of the posterior changes as the prior parameters change. This enables sensitivity analysis for small perturbations in the prior, providing insights into the robustness of the posterior estimates under minor changes in assumptions. It also allows solution continuation when dealing with significant alterations in prior beliefs, facilitating a comprehensive understanding of how large shifts in assumptions affect the posterior distribution. We focus on a class of non-conjugate hierarchical models tailored to encourage sparsity in linear inverse problems. The specific hierarchical model of interest is chosen since it is parameterized by a small number of shape parameters, and includes most classical sparsity promoting priors as special cases. As the shape parameters change, the posterior can transition continuously from a tractable unimodal distribution to an intractable multimodal distribution. To track the change in the posterior, we adopt particle based variational inference methods, specifically Stein Variational Gradient Descent (SVGD). SVGD iteratively updates a set of samples to minimize the KL-divergence away from a desired target distribution. We augment SVGD by Birth-Death sampling, which can efficiently exchange mass between separated modes, while simultaneously optimizing the kernel bandwidth used to derive the SVGD update. This method enables the discovery of new modes by tracing the modes as they branch out of a simpler, unimodal posterior, derived within the same family of priors.

Problem

• Bayesian inverse problems
• posterior sensitivity
• prior assumptions
• hierarchical models
• sparsity

Innovation

• Stein Variational Gradient Descent
• Birth-Death sampling
• Bayesian hierarchical model
• posterior sensitivity
• sparsity-promoting priors