PhiZero: A World Model Built Around Physical Language

2026-07-30

AI Summary

This work proposes a physics-informed world model that addresses the limitations of conventional video prediction approaches, which typically model dynamics directly in pixel space and struggle to capture underlying physical principles explicitly. The proposed method learns a compact, discrete "physics language" from unlabeled real-world videos through self-supervision, enabling explicit representation of world states. It adopts a "reason-then-render" paradigm: future states are predicted by performing action-conditioned sequential reasoning in this discrete latent space, followed by rendering to generate future video frames. This approach yields interpretable, physically consistent dynamics, demonstrating strong performance in both generative and perceptual tasks. Moreover, it supports interactive simulation, fine-grained action control, and zero-shot motion transfer, highlighting its capacity for structured and generalizable physical reasoning.

Abstract

We introduce PhiZero, a physical world model built around physical language, a compact discrete representation of world-state transitions. Existing physical world models typically predict future videos directly in pixel space, leaving the underlying world dynamics implicit within high-dimensional visual predictors. Motivated by humans' ability to abstract predictive structure from visual experience and organize it in natural language for explicit reasoning, we learn physical language from in-the-wild videos through self-supervision and use it to explicitly reason about how the physical world evolves. Accordingly, PhiZero adopts a reason-then-render paradigm: it first infers future world evolution as a physical-language sequence and then renders the inferred transitions into videos. Extensive experiments across generation and understanding benchmarks validate the ability of PhiZero to model physically coherent world evolution. We further show its potential for realistic and interactive world modeling, fine-grained action-conditioned simulation, and zero-shot motion transfer.

Problem

• physical world modeling
• world dynamics
• explicit reasoning
• discrete representation
• video prediction

Innovation

• physical language
• world model
• reason-then-render
• self-supervised learning
• discrete representation