TPD: Temporal Prior Decoupling for Text-to-Video Diffusion Models

2026-07-29

AI Summary

Text-to-video diffusion models often fail to generate key content in temporally evolving prompts—such as “a sandcastle being washed away by waves”—due to strong priors from early scenes suppressing later semantic signals. This work proposes a training-free, sampling-stage framework that constructs temporal counterfactual conditions containing only the early scene and introduces a lower-bound constraint in the classifier-free guidance space to recover the suppressed directional signal of later events, departing from conventional upper-bound rejection strategies. Combined with frame-selective and temporally joint optimization, the method significantly enhances the generation of later-stage events while preserving visual fidelity and early-frame coherence. The approach is compatible with multiple diffusion architectures, demonstrating the ubiquity of temporal prior suppression across models.

Abstract

Text-to-video diffusion models generate temporally coherent content from natural language, yet when a prompt describes an early scene that persists while a new event emerges on top of it---such as "a tall sandcastle standing on a beach where a wave rushes in and washes it away"---generation frequently fails to realize the late-segment event in the corresponding frames. We identify this failure as Temporal Prior Suppression (TPS): the dominant prior of the early segment captures the cross-attention trajectory across the temporal axis and suppresses the guidance signal needed for late-segment realization, a competing tendency existing guidance mechanisms do not model. We introduce Temporal Prior Decoupling (TPD), a training-free framework that restores suppressed late-segment signals during diffusion sampling. TPD constructs a temporal counterfactual by conditioning on the early segment alone, and defines the discrepancy between the full-prompt and counterfactual trajectories as a suppressed signal direction. Rather than removing this direction as in prior subtractive projection methods, TPD restores it through a frame-selective lower-bound constraint resolved jointly over diffusion timestep and video frame, realizing the suppressed event in the late frames without disrupting early-segment coherence: where prior work enforces upper-bound feasibility to remove unwanted semantics, TPD enforces lower-bound feasibility to guarantee suppressed-signal contribution. TPD runs entirely within standard diffusion sampling without retraining, and is defined purely in classifier-free guidance space, making it backbone-agnostic by construction. Experiments show that TPD significantly improves late-concept realization while preserving temporal coherence and visual fidelity, and that the targeted suppression recurs across distinct text-to-video backbones.

Problem

• Temporal Prior Suppression
• Text-to-Video Diffusion
• Late-segment Event Realization
• Temporal Coherence
• Guidance Signal Suppression

Innovation

• Temporal Prior Decoupling
• Temporal Prior Suppression
• Text-to-Video Diffusion
• Classifier-Free Guidance
• Counterfactual Conditioning