Temporal Concentration from Rollout Errors: Implicit Preference Optimization for Text-to-Video Diffusion

2026-07-30

AI Summary

This work addresses temporal sparse artifacts—such as motion collapse, object flickering, and color oversaturation—in text-to-video diffusion models, which stem from insufficient preference alignment. To mitigate these issues, the authors propose a centralized Implicit Preference Optimization (cIPO) framework that operates without human annotations or external reward models. cIPO leverages the forward noising and reconstruction process inherent in diffusion models to automatically generate implicit preference signals, then focuses optimization on high-error segments along the temporal dimension based on frame-level reconstruction errors. By integrating the iterative denoising mechanism of diffusion models with a temporally concentrated optimization strategy, cIPO significantly enhances video realism and temporal consistency across multiple datasets, effectively alleviating common temporal artifacts.

Abstract

Recent advances in preference alignment for diffusion-based video generation, particularly via Direct Preference Optimization (DPO), have significantly improved visual quality. However, temporally sparse artifacts such as motion collapse, object flickering, and color oversaturation remain a major barrier to perceptual realism. Existing methods struggle with these issues due to two key limitations: (1) the preference attribution bottleneck, where offline human annotations are costly and fail to accurately capture learning dynamics, while online reward signals are rollout-aware but often unstable and biased; and (2) temporal credit misallocation, where uniformly applied supervision cannot effectively target the brief segments in which artifacts occur. To address these challenges, we propose concentrated Implicit Preference Optimization (cIPO), a post-training framework for video diffusion models. cIPO derives implicit preference signals directly from the denoising process: given a real video, the model adds forward noise and reconstructs it via iterative denoising, treating the original as the preferred sample and the reconstruction as the dispreferred one. This formulation captures inference-time errors without requiring human annotations or external reward models. Moreover, frame-level discrepancies between original and reconstructed videos reveal when failures occur. cIPO leverages this by computing temporal reconstruction errors and concentrating optimization on high-error segments, enabling more precise correction of failure-prone regions. Extensive experiments demonstrate that cIPO consistently enhances video authenticity and temporal coherence across multiple datasets, highlighting the effectiveness and efficiency of implicit preference with temporally concentrated optimization.

Problem

• temporal artifacts
• preference alignment
• video diffusion
• credit misallocation
• perceptual realism

Innovation

• Implicit Preference Optimization
• Temporal Concentration
• Video Diffusion
• Preference Alignment
• Reconstruction Error