PhysAgent: Reflective Agentic Physics Control for Physically Plausible Video Generation

2026-07-17

AI Summary

Existing physics-guided video generation methods rely on single-pass predictions of physical parameters, which struggle to accurately capture user intent—particularly in modeling fine-grained dynamics, complex trajectories, and temporally coherent interactions. To address this limitation, this work proposes a reflective agent framework that treats physical programs as executable hypotheses and iteratively refines motion and interaction through a closed-loop “generate–simulate–verify–repair” process. The framework integrates a vision-language model, a physics simulation engine, and a dedicated control API to enable multi-stage user interaction and progressively realize precise event outcomes. Experimental results demonstrate that the generated videos significantly outperform existing approaches in terms of physical plausibility, alignment with input prompts, and generalization across diverse scenarios.

Abstract

Recent advances in physics-grounded video generation leverage physics simulation as a physical prior to guide video synthesis toward physically plausible outcomes. The simulation process is controlled by physical specifications, which are typically generated by a vision-language model in a single pass. Such one-shot prediction often fails to accurately translate user intent into executable simulations, particularly for fine-grained object dynamics, complex motion trajectories, and temporally structured interactions. In this paper, we propose PhysAgent, a reflective agentic framework that closes the loop among physical program generation, physics simulation, stage-specific verification, and targeted program repair. Beyond improving the control of coupled physical parameters, our framework enables the agent to progressively realize complex trajectories, multi-stage interactions, and precise event outcomes by treating each physical program as an executable hypothesis. In addition, we design a set of physics-control APIs to support more stable and complex motion behaviors. Extensive experiments demonstrate that PhysAgent produces more physically plausible videos, achieves better prompt alignment, and generalizes more effectively across diverse physical scenarios.

Problem

• physics-grounded video generation
• physical plausibility
• motion trajectory
• temporal interaction
• physical simulation control

Innovation

• reflective agentic
• physics simulation
• executable hypothesis
• physics-control API
• physically plausible video generation