Hessian Geometry of Latent Space in Generative Models

2025-06-12

AI Summary

Understanding the geometric structure of latent spaces in generative models—particularly diffusion models and statistical physics–inspired models—remains a fundamental challenge. Method: We propose learning the logarithm of the partition function via posterior approximation to construct an exponential-family Fisher information metric, thereby linking latent-space Hessian geometry to thermodynamic phase transition theory. Contribution/Results: We theoretically and empirically demonstrate the existence of fractal phase boundaries in latent space, where the Lipschitz constant diverges and the metric exhibits discontinuous transitions. Geodesic interpolation remains approximately linear within phases but fails across phase boundaries. Our approach outperforms baselines on Ising and TASEP models and successfully identifies and characterizes phase-transition structures in diffusion models. This work establishes a novel geometric framework for analyzing the intrinsic dynamics and generalization mechanisms of generative models, bridging statistical physics, differential geometry, and deep generative modeling.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Diffusion Models for Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

This paper presents a novel method for analyzing the latent space geometry of generative models, including statistical physics models and diffusion models, by reconstructing the Fisher information metric. The method approximates the posterior distribution of latent variables given generated samples and uses this to learn the log-partition function, which defines the Fisher metric for exponential families. Theoretical convergence guarantees are provided, and the method is validated on the Ising and TASEP models, outperforming existing baselines in reconstructing thermodynamic quantities. Applied to diffusion models, the method reveals a fractal structure of phase transitions in the latent space, characterized by abrupt changes in the Fisher metric. We demonstrate that while geodesic interpolations are approximately linear within individual phases, this linearity breaks down at phase boundaries, where the diffusion model exhibits a divergent Lipschitz constant with respect to the latent space. These findings provide new insights into the complex structure of diffusion model latent spaces and their connection to phenomena like phase transitions. Our source code is available at https://github.com/alobashev/hessian-geometry-of-diffusion-models.

Problem

• Analyzing latent space geometry in generative models
• Reconstructing Fisher information metric for exponential families
• Revealing fractal phase transitions in diffusion models

Innovation

• Reconstructs Fisher information metric for latent space
• Approximates posterior distribution of latent variables
• Reveals fractal phase transitions in diffusion models