langevin sampling

Efficient sampling from unnormalized Boltzmann densities remains challenging, particularly in high-dimensional multimodal distributions and Bayesian inference. This work proposes a probability flow ordinary differential equation (ODE) method based on linear stochastic interpolation, which, for the first time, employs a Langevin sampler to jointly draw samples from interpolated distributions and estimate the corresponding velocity field. This approach provides a theoretically grounded initialization and dynamic modeling for the flow ODE, ensuring stability and convergence. By integrating stochastic interpolation, probability flow ODEs, and Langevin diffusion, the method demonstrates efficient and robust sampling performance across a range of high-dimensional multimodal distributions and Bayesian inference tasks.

This work addresses linear Bayesian inverse problems in infinite-dimensional function spaces. We propose a preconditioned Langevin dynamics sampling method leveraging a score-based generative model (SGM) prior, ensuring algorithmic stability and convergence under mesh refinement. We rigorously define the infinite-dimensional Langevin sampler in Hilbert space for the first time and derive an explicit KL-divergence error bound. An optimal preconditioner—explicitly dependent on the score estimation error and the forward operator—is designed to achieve uniform convergence rates for multimodal posteriors. Theoretical analysis establishes sufficient conditions for global convergence in KL divergence, applicable to both Gaussian and generalized non-Gaussian priors. Numerical experiments demonstrate that the proposed preconditioner significantly enhances numerical stability and accelerates convergence.

Standard MCMC methods for Bayesian parameter estimation in ODEs suffer from high rejection rates, slow convergence, and excessive computational cost due to strong nonlinear dependencies among parameters and states. To address this, we propose a constraint-satisfying Langevin dynamics method that directly samples on the joint state-parameter manifold, embedding the ODE dynamics as hard constraints into the sampling process—thereby eliminating repeated forward numerical integration. Our approach uniquely integrates Langevin dynamics, trajectory optimization, and numerical continuation techniques within a constrained differential equation framework, enabling efficient posterior sampling while rigorously preserving the structural integrity of ODE solutions. Evaluated on a biochemical oscillator model, the method achieves over 100× improvement in sampling efficiency compared to standard MCMC. It accurately resolves Hopf bifurcations and characterizes limit-cycle regions, significantly enhancing uncertainty quantification and model selection capabilities.

To address the challenge of posterior sampling in high-dimensional, non-Gaussian Bayesian inverse problems where gradients are inaccessible, this paper proposes a gradient-free, affine-invariant ensemble sampling method. The core innovation couples second-order Langevin dynamics with Hamiltonian stochastic differential equations by introducing auxiliary momentum variables and designing a damping-driven mechanism, thereby constructing a novel stochastic dynamical system that preserves the target Gibbs measure. Furthermore, the method integrates covariance-adaptive preconditioning with ensemble averaging approximation to accelerate convergence without compromising invariance. This work establishes the first theoretical unification of second-order Langevin dynamics and ensemble approximation, significantly enhancing sampling efficiency and robustness. Extensive experiments on multiple high-dimensional Bayesian inverse problems demonstrate its superior performance over existing approaches.

A theoretical-practical gap persists in score-based diffusion models. Method: We propose a unified, reproducible SDE-based modeling framework that systematically integrates score matching, SDE/ODE solvers, denoising score estimation, and consistency modeling; notably, we introduce reinforcement learning into diffusion sampling for inference-path optimization. Contributions: (1) We establish theoretical consistency between sampling and score estimation under the SDE formulation; (2) we provide concise proofs of key theorems alongside practical algorithmic implementation guidelines; (3) we release modular, open-source code enabling rapid validation and extension to novel architectures. This work bridges the efficiency of score matching with scalable, RL-enhanced inference, delivering a foundational toolkit that balances theoretical rigor and engineering practicality for the design, analysis, and application of diffusion models.

本文通过结合算子学习和流方法，提出了一种神经采样器，用于从随机微分方程的不变测度中高效采样，显著提高了高维问题的处理速度。

This work addresses the irreducible sampling bias introduced by approximate composite score functions in compositional simulation-based inference. To mitigate this issue, the authors propose a solution based on annealed Langevin dynamics, modeling the composite score as a sequence of true scores associated with bridging densities and progressively controlling bias through iterative sampling. For the first time, the study provides theoretical guidance on key algorithmic parameters—including step size, number of iterations per annealing level, and total number of annealing levels—to guarantee a prescribed sampling accuracy. The analysis further reveals fundamental theoretical distinctions among different composite score constructions. By combining Wasserstein error bounds with closed-form solutions in Gaussian settings, the authors demonstrate that the construction by Linhart et al. is theoretically superior to that of Geffner et al. Empirical experiments corroborate the generalization efficacy of the proposed tuning strategy in complex inference scenarios.

Diffusion posterior samplers are widely used in inverse problems, yet their outputs suffer from bias and discretization instability at low temperatures, with the underlying mechanisms poorly understood. This work constructs a tractable surrogate path bridging the true posterior and a standard Gaussian distribution, leveraging the Feynman–Kac formula to express the density ratio as a path-space expectation. For the first time, it derives a partial differential equation that characterizes sampling bias. By integrating Radon–Nikodym derivatives, Ornstein–Uhlenbeck processes, and auxiliary drift reconstruction, the study reveals the origins and spatial distribution of bias in methods such as DPS and STSL: it precisely identifies regions of over- and under-sampling in DPS and explains how STSL enhances stability through a smoothed reaction term. This theoretical framework provides a foundation for designing stable and efficient posterior sampling algorithms.

This work proposes a unified variational generative modeling framework based on stochastic differential equations (SDEs) to efficiently address complex data generation tasks, including images, videos, and biomolecular structures. By incorporating both ordinary and stochastic differential equations, the authors derive the evidence lower bound (ELBO) from a variational inference perspective, systematically demonstrating that diffusion models, score matching, and flow matching are distinct parameterizations within this general framework. Through theoretical analysis grounded in the Fokker–Planck equation and empirical validation via one-dimensional density modeling experiments, the study provides clear comparisons among different parameterization strategies, confirming the proposed framework’s theoretical coherence, expressive capacity, and practical efficacy.

This work establishes a unified theoretical framework for diffusion models from the perspective of differential equations. Starting from a conditional Gaussian forward process, it derives the corresponding forward stochastic differential equation (SDE) and ordinary differential equation (ODE), and constructs a dynamical system that transports the data distribution to a standard Gaussian prior via marginalization. The framework then introduces a reverse SDE and a probability flow ODE, both driven by the marginal score function, thereby unifying score matching and noise prediction objectives. It rigorously demonstrates the equivalence of DDPM and DDIM in their training objectives while clarifying their fundamental distinction in sampling mechanisms—DDPM corresponds to a discretized reverse SDE, whereas DDIM implements a reverse ODE. Furthermore, the framework seamlessly incorporates mainstream sampling techniques such as DPM-Solver and classifier guidance, providing a coherent and rigorous continuous-time foundation for diffusion models.