An explicit formulation of the learned noise predictor $ε_θ({f x}_t, t)$ via the forward-process noise $ε_{t}$ in denoising diffusion probabilistic models (DDPMs)

2025-07-05

AI Summary

This work addresses the lack of theoretical understanding regarding the intrinsic relationship between the noise predictor $varepsilon_ heta(mathbf{x}_t, t)$ and the forward-process noise $varepsilon_t$ in Denoising Diffusion Probabilistic Models (DDPMs). We derive, for the first time, an explicit analytical expression of $varepsilon_ heta(mathbf{x}_t, t)$ in terms of $varepsilon_t$. Leveraging Markov chain modeling, variational inference, and score function theory—combined with stochastic differential equations and the Fokker–Planck equation—we rigorously prove the gradient-log-density identity. This establishes that the learned noise predictor fundamentally implements a weighted conditional expectation of the forward noise along the reverse trajectory. Our analysis provides a rigorous theoretical foundation for the core DDPM objective, significantly enhancing model interpretability and structural transparency.

Technology Category

Computer Vision: Diffusion Models for VisionMachine Learning: Probabilistic Circuits and Graphical ModelsReasoning under Uncertainty: Relational Probabilistic Models

Application Category

Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphsSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsSocial Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks

Abstract

In denoising diffusion probabilistic models (DDPMs), the learned noise predictor $ ε_θ ( {f x}_t , t)$ is trained to approximate the forward-process noise $ε_t$. The equality $ abla_{{f x}_t} log q({f x}_t) = -frac 1 {sqrt {1- {ar α}_t} } ε_θ ( {f x}_t , t)$ plays a fundamental role in both theoretical analyses and algorithmic design, and thus is frequently employed across diffusion-based generative models. In this paper, an explicit formulation of $ ε_θ ( {f x}_t , t)$ in terms of the forward-process noise $ε_t$ is derived. This result show how the forward-process noise $ε_t$ contributes to the learned predictor $ ε_θ ( {f x}_t , t)$. Furthermore, based on this formulation, we present a novel and mathematically rigorous proof of the fundamental equality above, clarifying its origin and providing new theoretical insight into the structure of diffusion models.

Problem

• Formulating noise predictor in DDPMs explicitly
• Linking forward-process noise to learned predictor
• Providing rigorous proof for fundamental equality

Innovation

• Explicit formulation of learned noise predictor
• Link between forward-process noise and predictor
• Rigorous proof of fundamental diffusion equality