🤖 AI Summary
This work addresses the desynchronization artifacts commonly observed in multimodal diffusion models due to their limited understanding of dynamic generative mechanisms. By formulating an analytically tractable framework based on coupled Ornstein-Uhlenbeck processes, the study integrates non-equilibrium statistical physics and spectral analysis to reveal that modalities stabilize sequentially—each governed by distinct characteristic timescales—rather than synchronously. The authors introduce the concept of a “synchronization gap” to quantify disparities in stabilization rates across modalities, and derive rigorous bounds linking coupling strength to symmetry-breaking stability, interpreting the system as a tunable temporal spectral filter. Experiments on MNIST demonstrate that time-dependent coupling schedules can precisely control modality-specific timescales, offering a theoretically grounded alternative to heuristic guidance strategies.
📝 Abstract
Diffusion based generative models have achieved unprecedented fidelity in synthesizing high dimensional data, yet the theoretical mechanisms governing multimodal generation remain poorly understood. Here, we present a theoretical framework for coupled diffusion models, using coupled Ornstein-Uhlenbeck processes as a tractable model. By using the nonequilibrium statistical physics of dynamical phase transitions, we demonstrate that multimodal generation is governed by a spectral hierarchy of interaction timescales rather than simultaneous resolution. A key prediction is the ``synchronization gap'', a temporal window during the reverse generative process where distinct eigenmodes stabilize at different rates, providing a theoretical explanation for common desynchronization artifacts. We derive analytical conditions for speciation and collapse times under both symmetric and anisotropic coupling regimes, establishing strict bounds for coupling strength to avoid unstable symmetry breaking. We show that the coupling strength acts as a spectral filter that enforces a tunable temporal hierarchy on generation. We support these predictions through controlled experiments with diffusion models trained on MNIST datasets and exact score samplers. These results motivate time dependent coupling schedules that target mode specific timescales, offering a potential alternative to ad hoc guidance tuning.