🤖 AI Summary
This study addresses the lack of first-order theoretical foundations and convergence guarantees in diffusion model sampling. By integrating stochastic differential equations (SDEs), Langevin dynamics, and non-convex optimization theory, it establishes a first-order analytical framework for diffusion models. The work reveals the contraction advantages of SDEs over ordinary differential equations (ODEs) and introduces a local score consistency certificate that does not require global convexity. Specifically, it proves that the reverse SDE flow exhibits exponential contraction in Fisher divergence under strongly convex potentials. Furthermore, it derives first-order stationarity bounds following discretization, yielding explicit exponential convergence rates and sampling convergence guarantees at the level of average gradient norms.
📝 Abstract
Recent literature has shown a strong connection between optimization and sampling. We develop the corresponding first-order theory for diffusion models. First, the SDE-based reverse-time flows of overdamped and underdamped Langevin diffusions contract relative Fisher divergences at explicit exponential rates whenever the stationary potential of the forward process is strongly convex---a condition on the noising process one chooses, not on the data. This is a unique advantage of SDE-based reverse diffusion, absent in the reverse process based on ODEs. Second, we incorporate discretization and establish averaged first-order stationarity bounds---the sampling analog of averaged gradient-norm guarantees in nonconvex optimization---for samplers of both overdamped and underdamped diffusion models. As in nonconvex optimization, the convexity-free certificate is local: it guarantees score consistency, not global mode weights.