Dimension-free error estimate for diffusion model and optimal scheduling

2025-12-01

AI Summary

Diffusion generative models suffer from dimensionality-dependent error quantification in high-dimensional settings: KL divergence requires absolute continuity, while Wasserstein distance bounds deteriorate exponentially with dimension. Method: We propose a dimension-agnostic error estimation framework, introducing smooth test functionals to derive dimension-independent upper bounds on the discrepancy between generated and target distributions. Leveraging time-reversed Ornstein–Uhlenbeck process simulation and neural score estimation, we formulate a variational optimization model to compute the optimal discretization time schedule minimizing approximation error. Contribution/Results: Theoretically, our bound remains stable regardless of dimension; empirically, the derived optimal scheduling significantly reduces generation bias. This provides tighter, more interpretable theoretical guarantees for high-dimensional diffusion modeling, advancing both foundational understanding and practical deployment.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Diffusion generative models have emerged as powerful tools for producing synthetic data from an empirically observed distribution. A common approach involves simulating the time-reversal of an Ornstein-Uhlenbeck (OU) process initialized at the true data distribution. Since the score function associated with the OU process is typically unknown, it is approximated using a trained neural network. This approximation, along with finite time simulation, time discretization and statistical approximation, introduce several sources of error whose impact on the generated samples must be carefully understood. Previous analyses have quantified the error between the generated and the true data distributions in terms of Wasserstein distance or Kullback-Leibler (KL) divergence. However, both metrics present limitations: KL divergence requires absolute continuity between distributions, while Wasserstein distance, though more general, leads to error bounds that scale poorly with dimension, rendering them impractical in high-dimensional settings. In this work, we derive an explicit, dimension-free bound on the discrepancy between the generated and the true data distributions. The bound is expressed in terms of a smooth test functional with bounded first and second derivatives. The key novelty lies in the use of this weaker, functional metric to obtain dimension-independent guarantees, at the cost of higher regularity on the test functions. As an application, we formulate and solve a variational problem to minimize the time-discretization error, leading to the derivation of an optimal time-scheduling strategy for the reverse-time diffusion. Interestingly, this scheduler has appeared previously in the literature in a different context; our analysis provides a new justification for its optimality, now grounded in minimizing the discretization bias in generative sampling.

Problem

• Deriving dimension-free error bounds for diffusion generative models
• Addressing limitations of Wasserstein and KL divergence in high dimensions
• Optimizing time scheduling to minimize discretization error in sampling

Innovation

• Dimension-free error bound using smooth test functionals
• Optimal time-scheduling strategy for reverse diffusion
• Functional metric for dimension-independent distribution guarantees