Efficient and Unbiased Sampling from Boltzmann Distributions via Variance-Tuned Diffusion Models

2025-05-27

AI Summary

To address Monte Carlo estimation bias in stochastic birth-death models (SBDMs) arising from inaccurate fractional derivative estimation, this paper proposes VT-DIS—a lightweight post-training method that achieves unbiased expectation estimation via variance-tuned backward-trajectory noise covariance adjustment. Its core contributions are: (i) the first application of α-divergence minimization (with α = 2) for forward–backward trajectory alignment; and (ii) the introduction of single-trajectory-level importance weights, which preserve unbiasedness while circumventing costly PF-ODE solvers—thereby substantially reducing computational overhead and mitigating dimensionality sensitivity. VT-DIS requires only a pre-trained SBDM and no retraining. Empirical evaluation on DW-4, LJ-13, and alanine-dipeptide benchmarks yields effective sample sizes of 80%, 35%, and 3.5%, respectively, at significantly lower computational cost than both PF-ODE and standard diffusion with importance sampling baselines.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Score-based diffusion models (SBDMs) are powerful amortized samplers for Boltzmann distributions; however, imperfect score estimates bias downstream Monte Carlo estimates. Classical importance sampling (IS) can correct this bias, but computing exact likelihoods requires solving the probability-flow ordinary differential equation (PF-ODE), a procedure that is prohibitively costly and scales poorly with dimensionality. We introduce Variance-Tuned Diffusion Importance Sampling (VT-DIS), a lightweight post-training method that adapts the per-step noise covariance of a pretrained SBDM by minimizing the $alpha$-divergence ($alpha=2$) between its forward diffusion and reverse denoising trajectories. VT-DIS assigns a single trajectory-wise importance weight to the joint forward-reverse process, yielding unbiased expectation estimates at test time with negligible overhead compared to standard sampling. On the DW-4, LJ-13, and alanine-dipeptide benchmarks, VT-DIS achieves effective sample sizes of approximately 80 %, 35 %, and 3.5 %, respectively, while using only a fraction of the computational budget required by vanilla diffusion + IS or PF-ODE-based IS.

Problem

• Corrects bias in Boltzmann distribution sampling via SBDMs
• Reduces computational cost of importance sampling in diffusion models
• Enhances efficiency of unbiased Monte Carlo estimates

Innovation

• Variance-Tuned Diffusion Importance Sampling (VT-DIS)
• Minimizes alpha-divergence for unbiased sampling
• Lightweight post-training with negligible overhead