scalable variational inference

This work addresses the core challenge of accurately inferring high-dimensional, non-convex posterior distributions in Bayesian neural networks and deep generative models. To this end, it systematically establishes the first comprehensive methodology framework for Bayesian approximate inference tailored to deep learning. The framework unifies variational inference, Markov chain Monte Carlo (including stochastic gradient samplers), Laplace approximation, and probabilistic programming into a coherent classification taxonomy and practical paradigm—thereby bridging Bayesian computation with modern deep architectures. The proposed methods substantially improve both posterior approximation accuracy and computational efficiency. Empirically validated across diverse deep Bayesian models, they deliver a theoretically principled yet engineering-practical inference toolkit for trustworthy AI systems.

This work addresses the computational intractability of posterior inference in Bayesian neural networks, which hinders scalability. Traditional sequential Monte Carlo (SMC) methods rely on full-batch data, incurring prohibitive computational costs. To overcome this limitation, the authors propose a data annealing strategy that incrementally incorporates mini-batches within the SMC framework, enabling progressive updates to the likelihood and gradient estimates. This approach represents the first effective integration of mini-batch processing with SMC sampling. By doing so, it achieves substantial gains in computational efficiency while preserving sampling accuracy. Empirical evaluations on standard image classification benchmarks demonstrate up to a six-fold speedup compared to conventional SMC, with negligible degradation in model accuracy.

To address the computational bottleneck in Bayesian inference arising from repeated evaluations of expensive black-box models, this paper introduces “post-hoc Bayesian inference”: a new paradigm that constructs posterior approximations solely from existing model evaluations—such as those collected along a maximum a posteriori (MAP) optimization trajectory—without any additional model queries. The core method, variational sparse Bayesian quadrature (VSBQ), is the first to unify sparse Gaussian process modeling, Bayesian quadrature, and variational inference, enabling efficient posterior estimation under noisy or black-box likelihoods. Evaluated on synthetic benchmarks and a real-world computational neuroscience application, VSBQ achieves posterior accuracy comparable to standard MCMC or variational inference (VI) methods, while accelerating inference by one to two orders of magnitude—all using only pre-existing optimization trajectory data. This significantly reduces the computational cost of Bayesian inference without sacrificing fidelity.

Gaussian variational approximation, while computationally efficient, fails to capture complex posterior structures—particularly skewness—especially in hierarchical Bayesian models. To address this, we propose a conditional skewness-corrected variational inference method: skewness correction is explicitly embedded into the variational family definition, enabling the first adaptive coupling of skewness parameters to local conditional posteriors. By decoupling location, scale, and skewness modeling, our approach avoids introducing extraneous global skewness parameters and supports joint optimization of both global and local parameters. Built upon a hierarchical variational family, the method incurs only marginal additional computational cost over standard Gaussian variational inference. Experiments on generalized linear mixed models and multinomial logit discrete choice models demonstrate substantial improvements in posterior approximation accuracy, effectively mitigating systematic bias arising from skewness neglect in conventional methods.

Traditional Approximate Bayesian Computation (ABC) methods rely on manually specified summary statistics, distance metrics, and tolerance thresholds, compromising the robustness and reproducibility of posterior inference. To address this, we propose ABCD-Conformal—the first fully automated, parameter-free ABC framework. It eliminates hand-crafted summary statistics and explicit distance computations by employing amortized neural likelihood-ratio estimation, augmented with Monte Carlo Dropout for principled uncertainty quantification. Crucially, we integrate conformal prediction to construct confidence sets for posterior moments—such as the posterior mean—with finite-sample, distribution-free frequentist coverage guarantees. Evaluated across four multivariate parameter inference tasks, ABCD-Conformal consistently outperforms state-of-the-art ABC methods in accuracy, calibration, and computational efficiency.

This work addresses the challenge of intractable posterior distributions in Bayesian inference, where traditional Markov chain Monte Carlo (MCMC) methods are computationally expensive and variational inference (VI) often sacrifices accuracy for efficiency. The paper presents the first systematic integration of VI and MCMC, introducing two novel algorithms: one leverages Gaussian VI to optimize the linear transformation matrix in Hamiltonian Monte Carlo (HMC), thereby enhancing sampling efficiency; the other combines variational autoencoders with Metropolis–Hastings sampling to improve exploration of multimodal posteriors. Experimental results demonstrate that the proposed methods significantly outperform standard MCMC on high-dimensional, complex distributions, effectively capturing all posterior modes while maintaining both computational efficiency and estimation accuracy.

This work addresses the computational inefficiency often encountered in enriched Dirichlet process mixture models within Bayesian nonparametric inference, particularly when employing complex MCMC algorithms or handling large-scale data. The authors propose an improved truncation approximation strategy integrated with variational Bayes, which substantially simplifies model implementation and accelerates inference. The resulting variational solution serves as a high-quality initialization for Gibbs sampling and is further enhanced by combining blocked Gibbs updates with Pólya urn sampling schemes, enabling efficient implementation within the Nimble platform. Experimental results demonstrate that the proposed approach achieves substantial gains in computational efficiency and practical usability while preserving inferential accuracy.

This work addresses the limitations of traditional variational inference—such as mean-field approximations, which often fail to capture posterior dependencies and thus yield inaccurate uncertainty quantification—while avoiding the prohibitive computational cost of Markov chain Monte Carlo (MCMC). The authors propose Variational Predictive Resampling (VPR), a novel approach that integrates variational inference within a predictive resampling framework. By iteratively simulating future observations, updating the variational approximation, and recording the corresponding parameters, VPR efficiently approximates the full Bayesian posterior. The method retains scalability while recovering posterior dependencies omitted by mean-field assumptions and is theoretically shown to converge to the true posterior in Gaussian location models. Empirical results demonstrate that VPR significantly improves the accuracy of posterior uncertainty quantification across multiple models, achieving computational efficiency comparable to or better than MCMC.

This study addresses the computational challenges and inefficiency associated with Bayesian inference for stochastic volatility mean models under heavy-tailed distributions. To overcome these limitations, the authors extend the model to the class of scale mixtures of normal distributions and develop a fast approximate Bayesian inference framework based on hidden Markov models. By leveraging special functions to circumvent numerical integration and incorporating parallel computing strategies, the proposed approach substantially enhances both algorithmic stability and computational efficiency. The method achieves inference accuracy comparable to that of conventional Markov chain Monte Carlo (MCMC) techniques while accelerating computation by approximately an order of magnitude, thereby offering an efficient and practical Bayesian solution for modeling high-dimensional financial time series.

This work proposes a variational Bayesian framework that jointly models the posterior and predictive distributions, circumventing the conventional two-stage pipeline of posterior approximation followed by Monte Carlo propagation. By leveraging amortized variational inference, the method completes training offline, thereby eliminating the need for repeated online sampling and significantly reducing computational overhead during prediction. The approach introduces a variational upper bound on the KL divergence together with a moment-matching regularization term to enable efficient and accurate uncertainty quantification. Evaluated on both analytical benchmarks and finite-element-based solid mechanics problems, the proposed method achieves substantially higher predictive accuracy compared to traditional approaches while drastically lowering online computational costs, making it well-suited for high-fidelity models where conventional Bayesian inference is prohibitively expensive.