Technology Category
Machine Learning: Learning with ManifoldsComputer Vision: Diffusion Models for VisionReasoning under Uncertainty: Relational Probabilistic Models
Analysis of Diffusion Models for Manifold Data
🤖 AI Summary
This work investigates the reverse-time dynamics of generative diffusion models on low-dimensional Gaussian mixture manifolds embedded in high-dimensional ambient space, focusing on characterizing the onset times of two phase transitions: “species differentiation” and “collapse.” We derive an exact mutual information (free energy) formula based on a generalized linear model and perform asymptotic analysis in the joint large-dimension and exponential-sample-size limit. This yields the first closed-form analytical expressions for phase transition times as functions of key geometric parameters—including the ratio of manifold dimension to ambient dimension. Our results establish a quantitative link between manifold geometry and diffusion dynamics, revealing the critical mechanism underlying the implicit modeling capability of diffusion models on nontrivial manifolds. The theory provides a foundational understanding of training and sampling dynamics for diffusion models applied to real-world low-dimensional data.
Technology Category
Application Category
📝 Abstract
We analyze the time reversed dynamics of generative diffusion models. If the exact empirical score function is used in a regime of large dimension and exponentially large number of samples, these models are known to undergo transitions between distinct dynamical regimes. We extend this analysis and compute the transitions for an analytically tractable manifold model where the statistical model for the data is a mixture of lower dimensional Gaussians embedded in higher dimensional space. We compute the so-called speciation and collapse transition times, as a function of the ratio of manifold-to-ambient space dimensions, and other characteristics of the data model. An important tool used in our analysis is the exact formula for the mutual information (or free energy) of Generalized Linear Models.
Problem
Research questions and friction points this paper is trying to address.
Analyze time reversed dynamics of diffusion models
Compute transitions in manifold data models
Determine speciation and collapse transition times
Innovation
Methods, ideas, or system contributions that make the work stand out.
Generative diffusion models analysis
Manifold data statistical modeling
Mutual information exact formula
🔎 Similar Papers
No similar papers found.
