proposal distribution design

Addressing the challenges of expensive, stochastic, and analytically intractable function evaluations in reinforcement learning (RL) and approximate Bayesian computation (ABC), this paper systematically reviews and refactors the Monte Carlo methodology framework. We first unify surrogate modeling approaches—designed for costly, noisy, and intractable densities—into three principled categories, and propose a modular surrogate modeling paradigm that jointly optimizes accuracy, computational cost, and robustness. Our framework is innovatively extended to likelihood-free inference and online RL settings. Integrating Bayesian optimization, Gaussian processes, sequential Monte Carlo, importance sampling, and adaptive experimental design, we conduct comprehensive numerical experiments to quantitatively characterize the trade-offs among sample efficiency, convergence stability, and noise robustness. The results provide a reusable, principled guideline for method selection in RL policy evaluation and hyperparameter optimization.

This study investigates the asymptotic behavior and computational efficiency of Piecewise Deterministic Monte Carlo (PDMC) algorithms—specifically the Bouncy Particle Sampler and the Zig-Zag Process—in complex settings characterized by high dimensionality, anisotropy, and large data volumes. By modeling these algorithms as continuous-time Markov processes and leveraging scaling limit analysis together with functional central limit theorems, the work systematically examines their theoretical properties. The analysis clarifies convergence characteristics and dimension dependence across a range of challenging scenarios, elucidating how anisotropy and dataset size influence algorithmic efficiency. These findings provide a rigorous theoretical foundation to guide the selection and refinement of PDMC methods in practical applications.

This study investigates the optimal scaling of high-dimensional Metropolised MCMC algorithms, focusing on how to adjust proposal distributions with increasing dimensionality to maintain sampling efficiency. Building upon the symmetry of the Metropolis–Hastings algorithm and high-dimensional asymptotic analysis, the authors develop a unified framework applicable to a broad class of target distributions and proposal mechanisms. The approach not only recovers classical results—such as the $O(1/d)$ variance scaling for Random Walk Metropolis (RWM) and $O(1/d^{1/3})$ for Metropolis-Adjusted Langevin Algorithm (MALA)—but also derives a novel class of gradient-driven MALA proposals with an optimal scaling law: their variance can be set to $O(1/d^\mu)$ for arbitrarily small $\mu > 0$, substantially outperforming existing methods. The theoretical analysis integrates non-product target measures and proposal distributions generated by implicit integrators of differential equations, demonstrating enhanced adaptability to dimensionality.

Stochastic MALA (sMALA) suffers from posterior drift—deviating from the true Gibbs posterior toward a non-Gibbs target—due to minibatch-induced bias, thereby compromising uncertainty quantification in Bayesian neural networks. Method: We propose a stochastic Metropolis–Hastings algorithm with an explicit, computable correction term that rigorously restores sampling from the original Gibbs posterior. This is the first MH-based stochastic sampler to incorporate an analytically tractable bias correction. Contributions/Results: We establish a PAC-Bayesian theoretical foundation for deep nonparametric regression, proving that the corrected algorithm achieves optimal posterior contraction rates and yields credible sets with guaranteed high coverage probability. Numerical experiments demonstrate that its uncertainty quantification performance matches classical MALA and significantly outperforms uncorrected sMALA, validating both statistical fidelity and practical efficacy.

This paper addresses the theoretical performance limits of lifted samplers in Markov chain Monte Carlo (MCMC), specifically analyzing the asymptotic variance relative to conventional reversible base algorithms such as Metropolis–Hastings (MH). Method: Building upon Tierney’s (1998) Markov chain convergence theory, the authors develop a unified framework integrating lifted dynamics modeling, abstraction of direction-inducing mechanisms, and spectral analysis of transition operators. Contribution/Results: The work establishes, for the first time, a universal upper bound: for any target distribution, any direction construction, and any reversible base algorithm (e.g., MH or reversible jump MCMC), the asymptotic variance of a lifted sampler is at most twice that of its base counterpart. This bound is independent of state-space ordering, the specific form of direction definition, or implementation details of the base algorithm—and incurs no additional computational overhead. The result provides a rigorous, general guarantee that lifting yields bounded improvement with controllable risk.

This work addresses the challenge of locally adapting hyperparameters—particularly trajectory length—in Hamiltonian Monte Carlo (HMC) when sampling from complex, high-dimensional target distributions. We propose the Gibbs-based Self-Tuning (GIST) framework, which dynamically updates key hyperparameters (e.g., trajectory length) at each HMC iteration via Gibbs sampling conditioned on the current position and momentum. GIST is the first method to integrate Gibbs sampling into HMC hyperparameter adaptation, unifying stochastic HMC, polynomial HMC, and the No-U-Turn Sampler (NUTS) under a single probabilistic framework. It further introduces a novel, locally adaptive trajectory-length strategy that outperforms NUTS. Theoretically, GIST ensures exact Hamiltonian dynamics simulation; empirically, it achieves superior sampling efficiency and convergence stability across diverse models—including ill-conditioned high-dimensional Gaussian targets—consistently surpassing NUTS in benchmark evaluations.

Unadjusted Hamiltonian Monte Carlo (HMC) and underdamped Langevin algorithms suffer from asymptotic bias in high-dimensional sampling due to numerical integration errors and lack automatic step-size adaptation. Method: This paper establishes, for the first time, a quantitative relationship between Hamiltonian energy error and asymptotic bias, and proposes the first black-box, provably bounded step-size adaptation scheme that eliminates the need for Metropolis–Hastings (MH) correction. Contribution/Results: The method rigorously controls asymptotic bias within any user-specified tolerance. Theoretical analysis—validated on Gaussian and canonical Bayesian models—confirms strong bias controllability. Empirical evaluation demonstrates several-fold speedup over MH-adjusted samplers in high dimensions, alongside markedly improved stability and overall performance. This breakthrough overcomes a key practical barrier to deploying unadjusted samplers in real-world applications.

This work proposes a novel framework that integrates diffusion models with physics-informed sequential Monte Carlo (SMC) to generate solutions of partial differential equations (PDEs) that satisfy physical constraints. For the first time, a physics-guided mechanism based on PDE residuals is embedded directly into the stochastic sampling process of the diffusion model, enabling dynamic enforcement of multi-physics and coupled PDE constraints during generation by jointly leveraging observational data and physical laws. Experimental results across multiple benchmark PDE systems and multi-physics coupling problems demonstrate that the proposed method significantly outperforms existing state-of-the-art generative models, achieving lower numerical errors while maintaining high solution fidelity. This approach establishes a new paradigm for scalable, high-accuracy generative PDE solvers.

Traditional rejection sampling methods suffer from unstable efficiency due to their geometrically distributed runtimes. This work proposes a general accept-complement approach that leverages the properties of log-concave densities and knowledge of their mode locations to generate random variates with deterministic constant-time complexity. The method applies universally to any log-concave distribution accessible via black-box evaluation, encompassing important families such as Gamma and Beta distributions. By enabling efficient “one-liner” sampling, it establishes a unified and practical framework for random number generation that overcomes the variability inherent in classical rejection sampling while maintaining broad applicability.

This work investigates whether standard continuous-time generative models can effectively capture the probability distribution of highly sparse, globally constrained discrete structures—such as Sudoku puzzles. Treating complete Sudoku grids as a discrete subset within a continuously relaxed space, we train flow-matching and score-based generative models, and systematically compare the performance of ODE-, SDE-, and DDPM-style sampling in both unconditional generation and constraint-satisfaction tasks. We demonstrate for the first time that such models can assign non-zero probability mass to globally constrained combinatorial structures and satisfy constraints through stochastic sampling. Empirical results show that score-based stochastic sampling is the most stable, while DDPM-style sampling achieves the highest efficacy. Although less sample-efficient than specialized solvers, our approach establishes the feasibility of using general-purpose probabilistic generative models as Sudoku solvers, thereby extending the applicability of continuous generative modeling to discrete reasoning problems.

This work addresses the challenge of efficiently sampling from non-uniform density functions over linearly constrained domains in Bayesian inverse problems, particularly when conventional gradient-based methods degrade near domain boundaries. The authors propose a novel Markov chain Monte Carlo (MCMC) algorithm that uniquely integrates higher-order geometric information—specifically, both gradient and curvature—of the target density into a Hit-and-Run proposal mechanism. By doing so, the method rigorously preserves sample feasibility while substantially enhancing sampling efficiency. Empirical evaluations demonstrate that the proposed approach consistently outperforms existing constrained and unconstrained samplers across a range of complex linear constraint settings, exhibiting superior robustness and adaptability.

This paper addresses the fundamental trade-off between statistical efficiency and computational cost in Monte Carlo (MC) algorithms. We propose the first systematic algorithm selection framework, integrating complexity analysis (upper/lower/tight bounds), probabilistic modeling, and gradient-assisted sampling to rigorously characterize time/space complexity boundaries and applicability domains of mainstream methods—including Metropolis–Hastings and Hamiltonian Monte Carlo (HMC). Theoretically, we establish limits on performance gains achievable via gradient incorporation and adaptive tuning. Furthermore, we introduce an AI-driven accuracy-efficiency co-optimization paradigm, featuring adaptive parameter tuning and intelligent scheduling. We construct a standardized benchmark evaluating performance–accuracy trade-offs across major MC methods. Empirical results show HMC achieves 3–5× speedup and 40% lower error over traditional methods on high-dimensional distributions. All benchmarks and evaluation tools are open-sourced for reproducibility.