unified diffusion modeling

The relative difficulty of denoising tasks across timesteps in diffusion models remains controversial. Method: This work systematically quantifies denoising difficulty per timestep, leveraging both the convergence behavior of denoising error and the relative entropy between true and predicted distributions—revealing that early (low-timestep) denoising is significantly more challenging. Building on this insight, we propose a “curriculum learning” paradigm: timesteps are clustered by difficulty and trained progressively in stages, with joint optimization of the noise schedule. Contribution/Results: Our approach departs from conventional parallel full-timestep training, requiring no architectural or loss-function modifications and remaining compatible with diverse diffusion model enhancements. Extensive experiments on unconditional generation, class-conditional generation, and text-to-image synthesis demonstrate substantial improvements in both model performance and convergence speed.

Generative diffusion models face a fundamental mismatch in discriminative tasks (e.g., referring image segmentation): their inherently tolerant generation mechanism conflicts with the strict accuracy requirements imposed on intermediate inference steps. This work systematically characterizes the non-uniform impact of denoising timesteps on perceptual quality—a previously unexplored insight. We propose three key innovations: (1) a timestep-aware loss that dynamically weights supervision across diffusion steps; (2) distribution-shift-robust data augmentation to enhance generalization; and (3) a prompt-driven interactive backward sampling mechanism enabling iterative refinement. All techniques operate via discriminative fine-tuning—no architectural modifications are required. Evaluated on depth estimation, referring image segmentation, and general perception benchmarks, our approach achieves state-of-the-art performance, significantly improving multimodal discriminative accuracy and interaction robustness.

Conditional image synthesis with diffusion models suffers from a lack of systematic understanding due to architectural complexity, task heterogeneity, and diverse conditional mechanisms. Method: This paper introduces the first unified taxonomy for conditional diffusion modeling, categorizing approaches by *where* conditioning is injected—either into the denoising network architecture or the sampling process—and formalizes three paradigmatic stages: training, reuse, and specialization. It further classifies six mainstream sampling-time conditioning strategies. Contributions: Based on a structured analysis of over 100 works, the paper establishes a comprehensive knowledge framework and open-sources an authoritative resource repository (GitHub Awesome-Conditional-Diffusion-Models). It identifies persistent bottlenecks—including limited generalization, inefficient inference, and coarse-grained control—and proposes principled directions toward scalable, modular, and fine-grained conditional modeling.

This work addresses the lack of a general framework for modeling time-varying latent states in existing generative models, which often rely on auxiliary stochastic processes that are difficult to sample. The authors propose a novel approach that treats observation generation as a deterministic mapping of a tractable Markov process, employing an image-space stochastic process generator whose one-time marginal distribution matches that of a target projected process. The key innovation lies in extending Generator Matching—previously limited to static latent variables—to time-varying latent processes for the first time. By integrating stochastic process theory, Markov projections, and flow matching techniques, the method establishes a unified generative modeling framework. This framework not only subsumes existing models with discrete latent processes as special cases but also accommodates a broader class of time-varying latent conditions while rigorously ensuring consistency between the generated and target marginal distributions.

While generative models produce high-fidelity samples, they remain unreliable for compositional generalization—i.e., synthesizing novel concept combinations outside the training distribution. This work systematically investigates the compositional generalization of conditional diffusion models in a controlled synthetic setting. We decouple and independently manipulate three key data properties—frequency, structural composition, and factor disentanglement—to construct a multi-dimensional out-of-distribution (OOD) evaluation framework. Our core findings reveal a multiplicative emergence of compositional capability: overall performance equals the product of subtask accuracies; the order of capability emergence is governed by data generative structure; and low-frequency compositions incur substantially higher optimization costs. This is the first empirical demonstration of nonlinear, emergent compositional generalization in diffusion models, establishing quantitative links among data structure, capability emergence, and optimization cost—providing a data-centric theoretical foundation and design principles for trustworthy generative compositional reasoning.

Existing diffusion synchronization methods rely on heuristic designs and require task-specific tuning, limiting their generalization. This work proposes a unified diffusion synchronization framework grounded in optimal control theory. At test time, it employs variational optimization to infer control variables that steer multiple diffusion trajectories toward a consistent output while remaining faithful to the pretrained prior. The approach provides the first principled theoretical foundation for diffusion synchronization and enables cross-modal cooperative generation without any additional training. Evaluated across three distinct tasks, the method substantially outperforms existing baselines, significantly improving both generation consistency and applicability.

研究揭示了去噪扩散模型隐含使用类似Transformer的注意力机制，提出基于此简化图像生成算法，减少训练时间和计算资源。

To address the inefficiency and instability in sampling caused by stochastic denoising in discrete diffusion models, this paper proposes the first deterministic denoising framework that requires neither model retraining nor continuous embeddings. Methodologically, it constructs a deterministic reverse transition process based on a Markov chain, integrating an enhanced herding algorithm and weak chaotic dynamics to enable deterministic trajectory evolution over discrete state spaces. The core contribution is the systematic introduction of deterministic reverse processes into discrete diffusion modeling—eliminating the inherent randomness and redundant iterations of conventional sampling. Experiments demonstrate significant improvements: up to 3.2× faster sampling and enhanced sample quality (FID reduced by 18.7%, BLEU increased by 2.4%) on both text and image generation tasks. Performance matches that of continuous diffusion models, establishing a novel paradigm for discrete generative modeling.

Existing theoretical analyses of diffusion generative models are fragmented and suffer from inconsistent notation, hindering unified understanding and principled development. Method: This work establishes a rigorous, unified mathematical framework grounded in fundamental properties of Gaussian distributions. It systematically derives the closed-form marginal distribution of the forward noising process, the analytical form of the reverse posterior, and the variational lower bound, ultimately yielding an optimization objective equivalent to noise prediction. Contribution/Results: The framework reveals the intrinsic equivalence between DDIM and rectified flow; provides a unified probabilistic interpretation of classifier-guided and classifier-free guidance; and integrates SDE/ODE formulations, the Fokker–Planck equation, flow matching, and multi-scale modeling—ensuring both theoretical coherence and practical implementability. Validated on mainstream models including Stable Diffusion, the framework enables efficient sampling and precise modeling while unifying disparate theoretical perspectives.

This work addresses the desynchronization artifacts commonly observed in multimodal diffusion models due to their limited understanding of dynamic generative mechanisms. By formulating an analytically tractable framework based on coupled Ornstein-Uhlenbeck processes, the study integrates non-equilibrium statistical physics and spectral analysis to reveal that modalities stabilize sequentially—each governed by distinct characteristic timescales—rather than synchronously. The authors introduce the concept of a “synchronization gap” to quantify disparities in stabilization rates across modalities, and derive rigorous bounds linking coupling strength to symmetry-breaking stability, interpreting the system as a tunable temporal spectral filter. Experiments on MNIST demonstrate that time-dependent coupling schedules can precisely control modality-specific timescales, offering a theoretically grounded alternative to heuristic guidance strategies.