Why Do Video Diffusion Models Violate Physics? Unveiling the Flaws in Attention Mechanisms

2026-09-20

AI Summary

研究揭示了视频扩散模型因注意力机制缺陷而违反物理规律的问题，并提出调整RoPE频率的方法来改善运动规划，增强生成视频的物理合理性。

Abstract

Despite impressive visual quality, state-of-the-art video diffusion models often generate content that violates real-world physical laws. While existing solutions rely on external priors or specialized data, we investigate the root cause by exploring the internal mechanisms of these models. Specifically, we present the first interpretability study on the ''motion planning'' process of text-to-video diffusion models, revealing how motion trajectories form during early denoising stages. Building upon the ''first shape, then details'' finding, we combine cross-attention trajectory patterns with causal head contributions to identify a specific subset of attention heads driving motion planning. Further, our self-attention analysis shows that Rotary Position Embedding (RoPE) induces excessive spatial attention decay. This causes early candidate regions to prematurely lock into physically implausible positions, suppressing reasonable trajectories in adjacent frames and triggering generation failure modes. To address this fundamental flaw, we propose a lightweight architectural modification that scales the frequency of RoPE across different denoising steps. This strategy reduces excessive attention decay, helping the model explore better candidate regions to establish coherent physical motion. Finally, training-free and training-based experiments confirm the effectiveness of our approach in enhancing the physical commonsense of generated videos.

Problem

• video diffusion models
• physics violation
• attention mechanisms
• motion planning

Innovation

• motion planning
• attention mechanisms
• Rotary Position Embedding (RoPE)
• denoising stages
• physical commonsense