Does Physics Live in the Activations? Localizing Physical Quantities in Video Diffusion Models

2026-10-02

AI Summary

This study investigates whether video generation models internalize physical principles. Leveraging rigid body dynamics simulation data, we employ linear probes to analyze the internal activations of Diffusion Transformers (DiTs), probing for encoded physical information within their representations. Our findings reveal that physical quantities can be linearly decoded with high accuracy during the early stages of denoising. Crucially, this physical information is not inherent in the input but is actively constructed throughout the denoising process, adhering to a mechanism of local storage and global computation while exhibiting cross-scenario extrapolation capabilities. This work demonstrates the existence of manipulable physical semantic directions within the DiT activation space, offering new perspectives for understanding and guiding physical consistency in video generation.

Abstract

Video generation models produce strikingly realistic sequences and are increasingly proposed as world models, yet recent benchmarks reveal pronounced deficits in their physical reasoning. This raises the question of whether these models internalize physical principles or merely reproduce familiar motion patterns. We address this by probing internal representations of video Diffusion Transformers (DiTs) for simulator-derived ground-truth physical quantities spanning kinematic motion and rigid-body dynamics under gravity and contact. We find that these quantities are linearly decodable with high accuracy early in the denoising process, substantially outperforming a baseline decoded directly from the model's own noised latents, indicating that the relevant physical information is actively constructed during denoising rather than already present in the input. Additionally, we show that activations at on-object tokens carry the relevant physical information and that quantities defined over multiple frames are readable from single latent frames. Hence, information is sharply localized within the token sequence and is computed globally but stored locally. The probes further show partial extrapolation, transferring to scene variations and object configurations outside their training regime, so what they read is not simply a correlate of the scenes they were fit on. When fitted directly in the full-resolution activation space, the probing directions can serve as steering vectors to change the model's output.

Problem

• Video Diffusion Models
• Physical Reasoning
• World Models
• Internal Representations
• Diffusion Transformers

Innovation

• Video Diffusion Transformers
• Physical Quantity Probing
• Activation Localization
• Steering Vectors
• Mechanistic Interpretability