🤖 AI Summary
This work addresses the challenge of intractable posterior distributions in Bayesian inference, where traditional Markov chain Monte Carlo (MCMC) methods are computationally expensive and variational inference (VI) often sacrifices accuracy for efficiency. The paper presents the first systematic integration of VI and MCMC, introducing two novel algorithms: one leverages Gaussian VI to optimize the linear transformation matrix in Hamiltonian Monte Carlo (HMC), thereby enhancing sampling efficiency; the other combines variational autoencoders with Metropolis–Hastings sampling to improve exploration of multimodal posteriors. Experimental results demonstrate that the proposed methods significantly outperform standard MCMC on high-dimensional, complex distributions, effectively capturing all posterior modes while maintaining both computational efficiency and estimation accuracy.
📝 Abstract
Bayesian statistics makes inference based on Bayes' theorem, but the posterior distribution of unknown parameters is typically analytically intractable. To estimate the posterior, two widely used numerical approximation methods are Markov Chain Monte Carlo (MCMC) and variational inference (VI). MCMC methods produce asymptotically exact samples but are computationally intensive, while VI methods are faster and more scalable but may lack accuracy. This paper proposes combining MCMC and VI to construct algorithms that leverage the strengths of both. The first proposed algorithm uses Gaussian variational inference (GVI) with various covariance structures to derive a linear transformation matrix for Hamiltonian Monte Carlo (HMC). This method improves the efficiency of HMC, particularly in high-dimensional and complex target distributions. The second algorithm combines a VI-based generative model, the variational auto-encoder (VAE), with the Metropolis-Hastings (MH) sampler. The resulting VAE-MH sampler is efficient and effectively traverses the parameter space, outperforming standard MCMC methods in identifying all modes of multi-modal distributions.