DistillAlign: Coordinating Mode Covering and Mode Seeking in Autoregressive Video Distillation

2026-07-29

AI Summary

This work addresses the disconnect between initialization and distribution matching in existing autoregressive video distillation methods, which leads to insufficient mode coverage and limited diversity in student models. From a distribution alignment perspective, the authors propose a joint distillation strategy that simultaneously optimizes mode coverage and mode seeking objectives. They introduce a distribution evaluation protocol to reveal coverage discrepancies that conventional visual metrics fail to capture. The method integrates Distribution Matching Distillation (DMD), consistency distillation, and precision–coverage assessment within a shared latent space, leveraging reverse KL divergence for improved alignment. Experiments demonstrate that the approach, using only the Wan-1.3B teacher model, surpasses baselines based on the significantly larger Wan-14B model, achieving notable gains in generation quality, distribution coverage, and output diversity.

Abstract

Existing autoregressive video distillation methods commonly adopt a Distribution Matching Distillation (DMD)-based multi-stage pipeline. However, they typically decouple the initialization and DMD stages -- which then pursue different target distributions -- and judge the intermediate student mainly by visual scores such as VBench. In this paper, we revisit this design from a distributional perspective. Given the mode-seeking nature of the distribution matching loss, a good initialization should match the mode coverage of the target DMD teacher, rather than merely pursuing high quality. To analyze this, we introduce a distributional evaluation protocol that measures precision and coverage between student and teacher distributions in a shared latent space. It exposes differences hidden by visual scores: some initializations reach high precision but low coverage, leading to suboptimal refinement, while mode-covering ones preserve broader support. Furthermore, even when the target distributions are aligned, DMD's reverse-KL objective can still drive the student toward high-probability teacher regions in late training, reducing coverage and diversity. To address this, we propose joint distillation, which combines DMD's mode-seeking objective with a Consistency Distillation-based mode-covering constraint. Experiments show that our method improves generation quality, coverage, and diversity; notably, even with a Wan-1.3B DMD teacher, it outperforms baselines refined with Wan-14B, underscoring the importance of distributional alignment in autoregressive video distillation.

Problem

• autoregressive video distillation
• distribution matching
• mode covering
• mode seeking
• distributional alignment

Innovation

• Distributional Alignment
• Mode Covering
• Mode Seeking
• Autoregressive Video Distillation
• Joint Distillation