Weighting Schedules Govern What and When Score-Based Generative Models Learn from Multimodal Data

2026-09-28

AI Summary

This study addresses the unclear impact of weighting schedules on the learning mechanisms of score-based generative models trained on multimodal data. To this end, it proposes a high-dimensional exact analysis framework that integrates theoretical derivations from non-equilibrium Gaussian mixture models with numerical simulations to systematically characterize the dynamic relationship between the signal-to-noise ratio and pattern formation time. The work reveals that weight scheduling governs both the rate and temporal hierarchy of feature acquisition, demonstrating that all features become learnable only in proximity to the pattern formation time. These findings are empirically validated across image and genomic generation tasks, providing critical theoretical guidance for designing optimal weight functions w(t).

Abstract

Score-based generative models generate new samples by integrating a time-dependent drift that carries Gaussian noise onto the target distribution. In practice this drift is modeled by a neural network, trained on a loss integrated over time $t$ with a weighting schedule $w(t)$. Along the backward dynamics, and for multi-modal distributions, trajectories commit to modes of the target within a narrow time window, the \textit{speciation time}. In this work, focusing on high-dimensional data, we decompose the integrated loss into its single-time contributions and analyze each at fixed signal-to-noise ratio $\Lambda(t)$: we show that $\Lambda(t)$ sets the rate at which each feature of a multimodal target - the mode directions and their relative weights - is acquired during training. Crucially, at high $\Lambda(t)$ all mode directions are acquired together, on a single timescale insensitive to their amplitudes, while the relative weights are not learned at all. Only near the speciation time, where $\Lambda(t)$ becomes of order one, do all features become learnable, each on its own timescale: the weights are acquired jointly with the directions, and the directions at rates set by their relative amplitudes. For models trained on time-integrated objectives, the learning dynamics is then governed by how much of the weighting effectively sits near the speciation time, which provides insights on $w(t)$ design choices. These results follow from an exact high-dimensional analysis of the training dynamics of unbalanced and hierarchical Gaussian mixtures. Numerical experiments on image and human genome haplotype generation recover the predicted hierarchy of learning timescales in more complex settings.

Problem

• score-based generative models
• weighting schedules
• multimodal data
• speciation time
• learning dynamics

Innovation

• Score-based generative models
• Weighting schedules
• Speciation time
• Training dynamics
• High-dimensional analysis