What's Inside Your Diffusion Model? A Score-Based Riemannian Metric to Explore the Data Manifold

2025-05-16

AI Summary

This work addresses the ambiguity in the implicit geometric structure of data manifolds learned by diffusion models. We propose a parameter-free Riemannian metric construction method grounded in the Stein score function, marking the first approach to directly translate diffusion-based score estimation into an intrinsic metric tensor on the underlying data manifold. Without explicit manifold modeling, our method naturally encodes the geometric properties of image manifolds within the ambient Euclidean space, enabling geometry-aware geodesic interpolation and extrapolation. Evaluated on Rotated MNIST and Stable Diffusion-generated images, our method yields significantly more semantically coherent and visually realistic generation paths: LPIPS decreases by 18.7%, FID improves by 12.3%, and KID drops by 15.1%. These results empirically validate the effectiveness and superiority of geometry-guided generative modeling.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Learning with Manifolds
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Recent advances in diffusion models have demonstrated their remarkable ability to capture complex image distributions, but the geometric properties of the learned data manifold remain poorly understood. We address this gap by introducing a score-based Riemannian metric that leverages the Stein score function from diffusion models to characterize the intrinsic geometry of the data manifold without requiring explicit parameterization. Our approach defines a metric tensor in the ambient space that stretches distances perpendicular to the manifold while preserving them along tangential directions, effectively creating a geometry where geodesics naturally follow the manifold's contours. We develop efficient algorithms for computing these geodesics and demonstrate their utility for both interpolation between data points and extrapolation beyond the observed data distribution. Through experiments on synthetic data with known geometry, Rotated MNIST, and complex natural images via Stable Diffusion, we show that our score-based geodesics capture meaningful transformations that respect the underlying data distribution. Our method consistently outperforms baseline approaches on perceptual metrics (LPIPS) and distribution-level metrics (FID, KID), producing smoother, more realistic image transitions. These results reveal the implicit geometric structure learned by diffusion models and provide a principled way to navigate the manifold of natural images through the lens of Riemannian geometry.

Problem

• Characterize intrinsic geometry of diffusion model data manifold
• Define Riemannian metric using Stein score for geodesics
• Improve interpolation and extrapolation of image distributions

Innovation

• Score-based Riemannian metric for data manifold
• Metric tensor preserves tangential distances
• Efficient geodesic computation for interpolation