Video-Mirai: Autoregressive Video Diffusion Models Need Foresight

2026-06-02

AI Summary

Existing causal video generation models rely solely on supervision from the current frame, which fails to preserve the long-term consistency of identity, layout, and motion, resulting in a representational planning gap. This work proposes a novel training strategy that introduces non-causal “foresight” supervision: a frozen non-causal encoder globally processes the full video rollout, and a lightweight predictor distills the stop-gradient foresight targets back into the causal state representations. Crucially, this approach enables future-frame supervision of causal representations without altering the inference architecture or violating causality constraints, thereby effectively bridging the planning gap. On VBench, it improves the overall 5-second generation score from 83.8 to 84.6; for 30-second ultra-long generation, subject and background consistency scores rise from 84.9 and 90.2 to 88.5 and 91.9, respectively.

Abstract

Causal video generators must predict from the past, but they need not learn only from it. In streaming autoregressive video diffusion, each emitted segment becomes a commitment that future segments must preserve. Standard training, however, only asks each causal state to explain the present. This creates what we call a representation-level planning gap: states that fit the current segment may discard identity, layout, and motion information needed for a consistent future. We introduce Video-Mirai, a training-only method that closes this gap without changing causal inference: the generator rolls out causally, a frozen foresight encoder reads the completed rollout non-causally, and a lightweight predictor distills the resulting stopped-gradient targets into causal states. Future frames supervise representations, never generator inputs. At inference, the encoder and predictor are discarded, leaving the original architecture, per-step FLOPs, and KV-cache behavior unchanged. Video-Mirai improves a strong Causal-Forcing baseline on 5-second VBench from 83.8 to 84.6 in terms of Total Score. On 30-second rollouts beyond the training horizon, subject consistency improves from 84.9 to 88.5 and background consistency from 90.2 to 91.9. Ablations identify future-conditioned targets as the key ingredient, and probes show that future frames become more decodable from current features. Causality should constrain inference, not representation supervision. Our study highlights that visual autoregressive models need foresight. Project page: https://y0uroy.github.io/Video-Mirai.

Problem

• autoregressive video generation
• causal modeling
• temporal consistency
• representation learning
• video diffusion

Innovation

• foresight supervision
• autoregressive video diffusion
• causal video generation
• representation distillation
• temporal consistency