Polynomial time sampling from log-smooth distributions in fixed dimension under semi-log-concavity of the forward diffusion with application to strongly dissipative distributions

2024-12-31

AI Summary

This work addresses efficient sampling from log-smooth, strongly dissipative distributions in fixed dimension. We propose the first polynomial-time stochastic sampling algorithm based on the forward Ornstein–Uhlenbeck process. Methodologically, we reduce log-smooth sampling entirely to a regularity estimation problem and establish that strongly dissipative distributions satisfy semi-log-convexity—a structural property that eliminates exponential dependencies (e.g., (e^{RL^2})) and enables exponential complexity improvement. Our algorithm employs a low-cost fractional estimator, integrates diffusion modeling with KL-divergence error control, and achieves theoretical complexity ( ext{poly}(d, L, alpha^{-1}, varepsilon^{-1})), specifically (O(d^7 L^{d+2} varepsilon^{-2(d+3)} (L + eta)^2 d^{2(d+1)})). This yields significant computational gains under moderate accuracy requirements.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

In this article we provide a stochastic sampling algorithm with polynomial complexity in fixed dimension that leverages the recent advances on diffusion models where it is shown that under mild conditions, sampling can be achieved via an accurate estimation of intermediate scores across the marginals $(p_t)_{tge 0}$ of the standard Ornstein-Uhlenbeck process started at the density we wish to sample from. The heart of our method consists into approaching these scores via a computationally cheap estimator and relating the variance of this estimator to the smoothness properties of the forward process. Under the assumption that the density to sample from is $L$-log-smooth and that the forward process is semi-log-concave: $- abla^2 log(p_t) succeq -eta I_d$ for some $eta geq 0$, we prove that our algorithm achieves an expected $epsilon$ error in $ ext{KL}$ divergence in $O(d^7L^{d+2}epsilon^{-2(d+3)} (L+eta)^2d^{2(d+1)})$ time. In particular, our result allows to fully transfer the problem of sampling from a log-smooth distribution into a regularity estimate problem. As an application, we derive an exponential complexity improvement for the problem of sampling from a $L$-log-smooth distribution that is $alpha$-strongly log-concave distribution outside some ball of radius $R$: after proving that such distributions verify the semi-log-concavity assumption, a result which might be of independent interest, we recover a $poly(R,L,alpha^{-1}, epsilon^{-1})$ complexity in fixed dimension which exponentially improves upon the previously known $poly(e^{RL^2}, L,alpha^{-1}, log(epsilon^{-1}))$ complexity in the low precision regime.

Problem

• Sampling Algorithm
• Smooth and Strongly Dissipative Distributions
• Computational Complexity Reduction

Innovation

• Forward Diffusion Sampling
• Polynomial Time Complexity
• Smooth and Strongly Dissipative Distributions