Mean Flow Distillation: Robust and Stable Distillation for Flow Matching Models

2026-06-09

AI Summary

Flow Matching models rely on iterative ODE-based sampling, which incurs high computational costs and struggles to meet real-time requirements. Existing distillation approaches often neglect the underlying manifold geometry, leading to unstable training, high variance, and degraded generation quality. To address these issues, this work proposes Mean Flow Distillation (MFD), which achieves exact distribution alignment by matching expected mean velocities. We introduce a novel perspective by modeling the distillation process as a temporal low-pass filter, effectively suppressing optimization noise and enhancing trajectory consistency. Theoretically, we establish the Mean Flow Matching theorem, proving that aligning mean velocities alone suffices to guarantee distributional consistency. Empirically, MFD enables high-quality single-step generation in high-dimensional tasks such as 4D occupancy prediction and text-to-image synthesis, achieving state-of-the-art performance.

Abstract

Flow Matching models have demonstrated strong performance across a wide range of generative tasks. However, their reliance on ODE-based iterative sampling incurs substantial computational overhead in inference, which limits their applicability in real-time scenes. While distillation is a promising solution, existing approaches largely borrow from diffusion-based score matching, often failing to exploit the intrinsic geometric structure of flows and suffering from training instability, high variance, and degraded generation quality. In this paper, we propose Mean Flow Distillation (MFD), a novel distillation framework tailored for flow matching models. We theoretically demonstrate that MFD acts as a temporal low-pass filter, effectively suppressing the high-frequency optimization noise inherent in variational score distillation (VSD) while ensuring global trajectory consistency. We further prove the Mean Flow Matching Theorem, establishing that matching expected average velocities is sufficient for strict distribution alignment. Empirically, on challenging tasks of high-dimensional manifolds including 4D occupancy forecasting and text-to-image generation, MFD achieves state-of-the-art performance, enabling high-fidelity single-step generation.

Problem

• Flow Matching
• distillation
• training instability
• computational overhead
• generation quality

Innovation

• Mean Flow Distillation
• Flow Matching
• Temporal Low-pass Filtering
• Distribution Alignment
• Single-step Generation