Conditional Residual Prediction: Improving Autoregressive Video Diffusion without a Bidirectional Teacher

2026-10-08

AI Summary

This study addresses the significant quality degradation in causal video diffusion models caused by error accumulation from reliance on historical frames. To mitigate this, we propose a Conditional Residual Prediction (CRP) mechanism that enables the model to perform residual supplementation with the current input as the primary signal and historical conditions as auxiliary guidance. By integrating image model initialization with a large-scale autoregressive architecture, we establish a training paradigm that eliminates the need for bidirectional teacher distillation. The resulting Optica model achieves high-quality 480p video generation using only 15 million video samples, attaining a VBench score of 82.78. This performance nearly closes the gap with bidirectional models, offering a simpler and more scalable approach to autoregressive video generation.

Abstract

Causal video diffusion models generate video autoregressively, which suits streaming, interactive, and long-video generation. Under standard training, however, they often yield lower generation quality than bidirectional models of the same size. Many existing approaches address this gap by initializing from or distilling a pretrained bidirectional teacher. We instead train a causal model from an image-model initialization, with no bidirectional video model at any stage. Because this path requires neither a large bidirectional teacher nor a complex distillation pipeline, it is simpler and more scalable. On this path, we find that a causal model trained on ground-truth history becomes strongly dependent on it, so that at inference errors in its own generated history propagate forward. We hypothesize that much of this dependence is unnecessary, because the current input already determines much of what the history provides. We propose Conditional Residual Prediction (CRP), a simple recipe for reducing a model's reliance on a condition: the model first predicts the target without the condition, and the condition may only add a residual on top of this prediction. Applied to history, CRP makes the model predict each chunk from the present as far as it can and use the past only for what the present cannot supply. In controlled experiments, CRP nearly closes the 6.14-point gap to a bidirectional model trained under the same setup. Scaling this recipe, we train Optica, a 2B-parameter causal video model that autoregressively generates 5-second 480p videos and reaches 82.78 on VBench with only about 15M training videos.

Problem

• causal video diffusion
• autoregressive generation
• error propagation
• history dependence

Innovation

• Conditional Residual Prediction
• Causal Video Diffusion
• Autoregressive Generation
• Teacher-free Training
• Error Propagation