🤖 AI Summary
This paper addresses the modeling challenge of score functions for high-dimensional data under variable transformations. It establishes, for the first time, a rigorous differential-geometric theory for score functions under smooth invertible transformations and derives a general transformation formula. Methodologically: (1) it enables cross-space diffusion sampling via a reverse Itô lemma, decoupling spatial dependencies between forward and reverse processes; (2) it generalizes sliced score matching to arbitrary smooth transformations, yielding generalized sliced score matching. Key contributions include: the first principled separation of training in the original space from efficient sampling in the transformed space; stable diffusion modeling on constrained manifolds such as the probability simplex; and empirical results demonstrating that generalized sliced score matching significantly outperforms conventional linear-projection-based methods in high-dimensional density estimation.
📝 Abstract
We derive a general change of variables formula for score functions, showing that for a smooth, invertible transformation $mathbf{y} = phi(mathbf{x})$, the transformed score function $
abla_{mathbf{y}} log q(mathbf{y})$ can be expressed directly in terms of $
abla_{mathbf{x}} log p(mathbf{x})$. Using this result, we develop two applications: First, we establish a reverse-time It^o lemma for score-based diffusion models, allowing the use of $
abla_{mathbf{x}} log p_t(mathbf{x})$ to reverse an SDE in the transformed space without directly learning $
abla_{mathbf{y}} log q_t(mathbf{y})$. This approach enables training diffusion models in one space but sampling in another, effectively decoupling the forward and reverse processes. Second, we introduce generalized sliced score matching, extending traditional sliced score matching from linear projections to arbitrary smooth transformations. This provides greater flexibility in high-dimensional density estimation. We demonstrate these theoretical advances through applications to diffusion on the probability simplex and empirically compare our generalized score matching approach against traditional sliced score matching methods.