🤖 AI Summary
To address the challenge of sampling from high-dimensional, multimodal Bayesian posteriors, this paper proposes the Annealed Leap-Point Sampler (ALPS). Methodologically, ALPS introduces a novel “annealed” state-space augmentation: it constructs an annealed cold-state target distribution, locates modes via Laplace approximation, and enables efficient inter-modal transitions in the ultra-cold regime using Gaussian mixture–based independent Metropolis–Hastings. A temperature ladder then transfers this cross-modal capability to the original posterior. Theoretically, we prove that the coldest inverse temperature required scales only as *O*(*d*), and the overall algorithmic complexity is *O*(*d*³), markedly improving upon existing parallel tempering methods. Empirical evaluation on a high-dimensional multimodal SUR longitudinal model demonstrates ALPS’s robust inter-modal sampling performance, scalability to high dimensions, and guaranteed convergence.
📝 Abstract
In Bayesian statistics, exploring multimodal posterior distribution poses major challenges for existing techniques such as Markov Chain Monte Carlo (MCMC). These problems are exacerbated in high-dimensional settings where MCMC methods typically rely upon localised proposal mechanisms. This paper introduces the Annealed Leap-Point Sampler (ALPS), which augments the target distribution state space with modified annealed (cooled) target distributions, in contrast to traditional approaches which have employed tempering. The temperature of the coldest state is chosen such that its corresponding annealed target density can be sufficiently well-approximated by a Laplace approximation. As a result, a Gaussian mixture independence Metropolis-Hastings sampler can perform mode-jumping proposals even in high-dimensional problems. The ability of this procedure to"mode hop"at this super-cold state is then filtered through to the target state using a sequence of tempered targets in a similar way to that in parallel tempering methods. ALPS also incorporates the best aspects of current gold-standard approaches to multimodal sampling in high-dimensional contexts. A theoretical analysis of the ALPS approach in high dimensions is given, providing practitioners with a gauge on the optimal setup as well as the scalability of the algorithm. For a $d$-dimensional problem the it is shown that the coldest inverse temperature level required for the ALPS only needs to be linear in the dimension, $mathcal{O}(d)$, and this means that for a collection of multimodal problems the algorithmic cost is polynomial, $mathcal{O}left(d^{3}
ight)$. ALPS is illustrated on a complex multimodal posterior distribution that arises from a seemingly-unrelated regression (SUR) model of longitudinal data from U.S. manufacturing firms.