🤖 AI Summary
This work addresses the challenge in Flow Matching models where fixed time-step sampling strategies, such as midpoint biasing, struggle to balance training efficiency and sample quality. The study reframes time-step sampling as a dynamic curriculum and reveals that the loss landscape exhibits a U-shaped difficulty distribution across time steps. To exploit this insight, the authors propose a two-stage curriculum sampling strategy: initially employing midpoint-biased sampling to accelerate structural learning, followed by a switch to uniform sampling to refine boundary details. Evaluated on CIFAR-10, the method improves the Fréchet Inception Distance (FID) from 3.85 to 3.22 and achieves peak performance within 100,000 training steps—significantly outpacing the 150,000 steps required by uniform sampling.
📝 Abstract
Timestep sampling $p(t)$ is a central design choice in Flow Matching models, yet common practice increasingly favors static middle-biased distributions (e.g., Logit-Normal). We show that this choice induces a speed--quality trade-off: middle-biased sampling accelerates early convergence but yields worse asymptotic fidelity than Uniform sampling. By analyzing per-timestep training losses, we identify a U-shaped difficulty profile with persistent errors near the boundary regimes, implying that under-sampling the endpoints leaves fine details unresolved. Guided by this insight, we propose \textbf{Curriculum Sampling}, a two-phase schedule that begins with middle-biased sampling for rapid structure learning and then switches to Uniform sampling for boundary refinement. On CIFAR-10, Curriculum Sampling improves the best FID from $3.85$ (Uniform) to $3.22$ while reaching peak performance at $100$k rather than $150$k training steps. Our results highlight that timestep sampling should be treated as an evolving curriculum rather than a fixed hyperparameter.