Rollout-Marginal Distillation for Long-Horizon Autoregressive Video Generation

2026-09-29

AI Summary

This study addresses error accumulation in autoregressive video generation that degrades long sequences, as well as the weakened visual quality signals caused by context coupling in existing distillation methods. To tackle these issues, this work proposes the RMD framework, whose core innovation lies in decoupling visual quality from temporal consistency optimization. Specifically, it first preserves historical context while independently scoring each video chunk to precisely rectify visual quality and prevent artifact interference. Subsequently, video-level distribution matching distillation (DMD) is applied to restore temporal coherence. Experimental results demonstrate that the proposed approach maintains high visual quality and stability over durations far exceeding the training length, significantly outperforming conventional video-level DMD baselines.

Abstract

Autoregressive (AR) video diffusion enables low-latency, streamable video generation, but prediction errors often accumulate over long rollouts. Training the generator on its own rollouts exposes it to these imperfect histories. However, existing video-level distribution matching distillation (DMD) scores the whole rollout jointly. Because a chunk is evaluated together with its past and future, its correction can favor matching artifacts in the surrounding context merely to preserve temporal consistency. To provide a clearer visual-quality signal, we introduce Rollout-Marginal Distillation (RMD). RMD retains the generated history for AR prediction but scores each chunk independently against a chunk teacher, ensuring its quality correction is not compromised by an imperfect temporal context. To compensate for the lack of temporal context in independent chunk scoring, RMD subsequently applies video-level DMD to restore temporal coherence. Extensive experiments demonstrate that RMD maintains high visual quality far beyond its training horizon and outperforms video-level DMD baselines. Code and video results are available at https://cjeen.github.io/RMD

Problem

• autoregressive video generation
• error accumulation
• distribution matching distillation
• visual quality
• temporal consistency

Innovation

• Rollout-Marginal Distillation
• Autoregressive Video Generation
• Distribution Matching Distillation
• Long-Horizon Video
• Chunk-level Scoring