On randomized step sizes in Metropolis-Hastings algorithms

2026-01-27

AI Summary

This work addresses the high sensitivity of the Metropolis–Hastings (MH) algorithm to step size, which can severely degrade sampling efficiency when poorly tuned. To enhance robustness against step-size misspecification, the authors propose two randomized step-size strategies—one based on auxiliary variables and another via marginalization. Theoretical analysis shows that these randomized schemes preserve favorable properties of fixed-step kernels under weak Poincaré inequalities and maintain a spectral gap. Notably, the marginalized kernel reduces asymptotic variance and improves the optimal acceptance rate for both Langevin and Hamiltonian Monte Carlo samplers. Even under substantial step-size misspecification, the proposed methods retain a polynomially decaying spectral gap and scalability in high dimensions. Empirical evaluations on benchmark targets—including Poisson regression, Neal’s funnel, and the Rosenbrock distribution—demonstrate their robustness and computational efficiency.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Kernel Methods

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

The performance of Metropolis-Hastings algorithms is highly sensitive to the choice of step size, and miss-specification can lead to severe loss of efficiency. We study algorithms with randomized step sizes, considering both auxiliary-variable and marginalized constructions. We show that algorithms with a randomized step size inherit weak Poincar\'e inequalities/spectral gaps from their fixed-step-size counterparts under minimal conditions, and that the marginalized kernel should always be preferred in terms of asymptotic variance to the auxiliary-variable choice if it is implementable. In addition we show that both types of randomization make an algorithm robust to tuning, meaning that spectral gaps decay polynomially as the step size is increasingly poorly chosen. We further show that step-size randomization often preserves high-dimensional scaling limits and algorithmic complexity, while increasing the optimal acceptance rate for Langevin and Hamiltonian samplers when an Exponential or Uniform distribution is chosen to randomize the step size. Theoretical results are complemented with a numerical study on challenging benchmarks such as Poisson regression, Neal's funnel and the Rosenbrock (banana) distribution.

Problem

• Metropolis-Hastings algorithms
• step size
• algorithmic efficiency
• tuning sensitivity
• randomized step sizes

Innovation

• randomized step size
• Metropolis-Hastings
• spectral gap
• asymptotic variance
• robustness to tuning