propose-and-project sampling

This study investigates the optimal scaling of high-dimensional Metropolised MCMC algorithms, focusing on how to adjust proposal distributions with increasing dimensionality to maintain sampling efficiency. Building upon the symmetry of the Metropolis–Hastings algorithm and high-dimensional asymptotic analysis, the authors develop a unified framework applicable to a broad class of target distributions and proposal mechanisms. The approach not only recovers classical results—such as the $O(1/d)$ variance scaling for Random Walk Metropolis (RWM) and $O(1/d^{1/3})$ for Metropolis-Adjusted Langevin Algorithm (MALA)—but also derives a novel class of gradient-driven MALA proposals with an optimal scaling law: their variance can be set to $O(1/d^\mu)$ for arbitrarily small $\mu > 0$, substantially outperforming existing methods. The theoretical analysis integrates non-product target measures and proposal distributions generated by implicit integrators of differential equations, demonstrating enhanced adaptability to dimensionality.

This work addresses the challenge of efficiently sampling from non-uniform density functions over linearly constrained domains in Bayesian inverse problems, particularly when conventional gradient-based methods degrade near domain boundaries. The authors propose a novel Markov chain Monte Carlo (MCMC) algorithm that uniquely integrates higher-order geometric information—specifically, both gradient and curvature—of the target density into a Hit-and-Run proposal mechanism. By doing so, the method rigorously preserves sample feasibility while substantially enhancing sampling efficiency. Empirical evaluations demonstrate that the proposed approach consistently outperforms existing constrained and unconstrained samplers across a range of complex linear constraint settings, exhibiting superior robustness and adaptability.

Bayesian posterior sampling on Riemannian manifolds—such as Stiefel and Grassmann manifolds—is challenging, especially under anisotropic target densities; existing gradient-based or preconditioning-dependent methods suffer from poor robustness. Method: We propose the first geodesic-based slice sampling MCMC algorithm for Riemannian manifolds, generalizing Euclidean hit-and-run slice sampling by replacing straight-line segments with geodesics. Our method is gradient-free, requires no pre-tuning, satisfies detailed balance, and is geometrically adaptive. Contribution/Results: We establish theoretical guarantees of ergodicity and detailed balance. Empirical evaluation on synthetic and real-world data demonstrates faster convergence and superior mixing compared to state-of-the-art manifold MCMC methods. Crucially, our algorithm remains stable and efficient even under highly anisotropic posteriors, significantly enhancing the practicality and robustness of Bayesian inference on matrix manifolds.

Stochastic MALA (sMALA) suffers from posterior drift—deviating from the true Gibbs posterior toward a non-Gibbs target—due to minibatch-induced bias, thereby compromising uncertainty quantification in Bayesian neural networks. Method: We propose a stochastic Metropolis–Hastings algorithm with an explicit, computable correction term that rigorously restores sampling from the original Gibbs posterior. This is the first MH-based stochastic sampler to incorporate an analytically tractable bias correction. Contributions/Results: We establish a PAC-Bayesian theoretical foundation for deep nonparametric regression, proving that the corrected algorithm achieves optimal posterior contraction rates and yields credible sets with guaranteed high coverage probability. Numerical experiments demonstrate that its uncertainty quantification performance matches classical MALA and significantly outperforms uncorrected sMALA, validating both statistical fidelity and practical efficacy.

Metropolis–Hastings samplers—especially informed variants—exhibit slow convergence and dimension-dependent relaxation times in high-dimensional discrete parameter spaces under multimodal posteriors. Method: We develop the first dimension-free mixing time theory for information-driven MCMC on discrete domains, integrating multicommodity flow analysis with single-site drift conditions and leveraging high-dimensional statistical structure to derive verifiable sufficient conditions. Contribution/Results: Our analysis yields a tight, dimension-independent upper bound on the relaxation time, breaking the conventional dimensional dependence bottleneck in convergence analysis. This provides the first theoretically grounded, computationally efficient sampling framework for discrete-parameter inference tasks—such as Bayesian model selection—where scalability with dimension is critical. The bound is constructive and applicable to a broad class of informed discrete MCMC algorithms, enabling rigorous performance guarantees without restrictive assumptions on posterior geometry or sparsity.

This work addresses the inefficiency and lack of a unified theoretical understanding of gradient-based MCMC methods when sampling from anisotropic and hierarchical posteriors. Starting from continuous-time dynamics, the authors systematically derive algorithms such as HMC, MALA, NUTS, and MAKLA through numerical discretization and Metropolis correction, establishing a cohesive theoretical framework. They introduce two key innovations: a globally whitened mass matrix and a stochastic step-size strategy, which together mitigate sampling difficulties arising from state-dependent curvature. The proposed approach substantially improves sampling efficiency and demonstrates superior convergence and stability in large-scale Bayesian inference and hierarchical models.

This work addresses the challenge of efficiently sampling from feasible regions composed of multiple disconnected components whose number is unknown a priori. The authors propose MASEM, a novel method that, for the first time, maximizes empirical entropy over implicitly constrained manifolds without requiring prior knowledge of the number of connected components. MASEM integrates k-nearest-neighbor density estimation, a resampling mechanism, and multiple local samplers in a coordinated framework, and it comes with theoretical guarantees of exponential convergence. Experimental results demonstrate that MASEM achieves an order-of-magnitude improvement in Sinkhorn distance over existing approaches on both synthetic and robotic benchmark tasks, while maintaining competitive computational efficiency.

This work addresses the slow mixing and vanishing gradient issues of conventional MCMC methods when sampling heavy-tailed distributions in high-dimensional unbounded spaces. The authors propose a novel approach based on radial diffeomorphic contraction, which maps the target distribution into the unit ball and leverages efficient interior-ball random walk algorithms—such as Ball Walk—on this convex domain. To further enhance efficiency, variational inference is employed to pre-tune a spherical self-diffeomorphism that approximately enforces log-concavity. This framework constitutes the first unified ergodic MCMC sampler capable of handling arbitrary polynomially decaying heavy-tailed distributions, accompanied by non-asymptotic rapid mixing guarantees. Empirical evaluations demonstrate substantial improvements over both the No-U-Turn Sampler and existing spherical projection-based samplers on real-world posterior benchmarks from PosteriorDB and synthetic high-dimensional heavy-tailed targets.

Efficient sampling from multimodal, unnormalized density functions on Riemannian manifolds remains challenging, as existing methods often fail to simultaneously respect the underlying geometric structure and capture complex distributional characteristics. This work proposes a training-free sampling framework that extends the principles of diffusion models to Riemannian manifolds for the first time. By constructing geometrically compatible stochastic interpolation paths and coupling them with nonequilibrium deterministic dynamics, the method gradually transports an easily sampled noise distribution toward the target distribution. Relying solely on standard Monte Carlo techniques and incorporating iterative posterior sampling, the approach demonstrates strong empirical performance in high-dimensional, heavy-tailed, and multimodal settings, offering both theoretical rigor and broad applicability.

This work addresses the challenge of inefficient posterior exploration in hierarchical discrete models with latent variables, where conventional MCMC methods struggle due to the need to integrate out latent variables. The authors propose a similarity-driven MCMC approach that constructs a proposal mechanism based on a data-driven measure of discrepancy between observations and model predictions, thereby guiding transitions toward regions of higher posterior support without explicitly integrating latent variables. This method represents the first application of similarity-driven proposals to discrete-space MCMC and is naturally suited to complex hierarchical discrete models. Experiments on both synthetic and real-world data demonstrate substantial improvements in sampling efficiency and posterior exploration, confirming its effectiveness in models such as Dirichlet–Multinomial regression.