🤖 AI Summary
In parallel tempering Markov chain Monte Carlo (PT-MCMC), temperature parameters are typically set manually, making it difficult to adapt to complex multimodal distributions. Method: This paper introduces policy gradient reinforcement learning—specifically the REINFORCE algorithm—into temperature scheduling for the first time, proposing an end-to-end differentiable, online adaptive temperature optimization framework. Leveraging the temperature-chain swap mechanism, the method dynamically adjusts temperatures across chains using integrated autocorrelation time (IACT) as the reward signal. Contribution/Results: Compared to conventional geometric spacing or uniform acceptance-rate schemes, our approach significantly reduces IACT on standard multimodal benchmarks, enhancing sampling mixing efficiency and inter-chain independence. It breaks away from fixed-schedule paradigms, enabling data-driven, fully automated temperature parameter optimization.
📝 Abstract
Parallel tempering is a meta-algorithm for Markov Chain Monte Carlo that uses multiple chains to sample from tempered versions of the target distribution, enhancing mixing in multi-modal distributions that are challenging for traditional methods. The effectiveness of parallel tempering is heavily influenced by the selection of chain temperatures. Here, we present an adaptive temperature selection algorithm that dynamically adjusts temperatures during sampling using a policy gradient approach. Experiments demonstrate that our method can achieve lower integrated autocorrelation times compared to traditional geometrically spaced temperatures and uniform acceptance rate schemes on benchmark distributions.