🤖 AI Summary
This work addresses the limitations of existing sampler warm-up strategies, which rely on fixed schedules and heuristic hyperparameter tuning and often fail to adapt to complex posterior geometries. The authors propose a general adaptive warm-up framework that reformulates warm-up as an evidence-based path selection problem. Guided uniformly by an estimated posterior covariance Σ_π under a fixed computational budget, the method dynamically adjusts hyperparameters in real time. Applicable to a broad class of Hamiltonian Monte Carlo (HMC) samplers, it integrates local metric adaptation with global geometric awareness, featuring path attractivity and an explicit rejection mechanism to avoid spurious claims of coverage. Experiments on ill-conditioned test cases and the German Credit dataset demonstrate that, compared to Fisher-informed low-rank or diagonal preconditioning strategies, the proposed approach achieves 1.13–2.45× higher gradient-normalized effective sample size (ESS) at equivalent posterior quality.
📝 Abstract
Sampler transitions are local, but their efficiency is governed by global posterior geometry. Warmup bridges the two by tuning the step size, metric, and related controls. Although these adaptation mechanisms are individually well studied, their orchestration is still commonly encoded through fixed schedules and heuristic choices: users preselect the metric family, compute budget, and response to incompatible geometry. We recast that orchestration as evidence-based routing within a declared budget. The Universal Warmup Path uses one sampler-independent compass, $Σ_π=\operatorname{Cov}_π(X)$, while allowing each route to deploy its own metric. Its discipline---gather evidence, act, wait, or refuse---is sampler-independent; we evaluate one gradient-based HMC-family realization using scalar gates. The confidence-aware design gives route-attractor guarantees, while a finite-transcript bound limits what confined evidence can establish. When local and global evidence agree, the path deploys an efficient local metric; disagreement fails loudly through reparameterization advice or population handoff, never certifying global coverage. Across the cells we tested, automatic warmup outperformed the predeclared Fisher low-rank primary, with geometric-mean ESS-per-gradient ratios of $1.409$--$2.451$ on the ill-conditioned suite and $1.131$--$1.951$ on German credit. It also improved ESS per gradient over the predeclared fixed-schedule diagonal control while meeting the same declared post-sampling quality criterion. The proposed path points toward a coherent theory of sampler hyperparameter tuning, connecting local adaptation, global geometry, and explicit refusal in practical controllers.