Learning Explicit Physical Parameter Control and Benchmarking for Video Generation

2026-07-21

AI Summary

This work addresses the challenge of achieving explicitly physics-driven controllable dynamics in image-to-video generation, particularly the confusion of physical attributes in multi-object interactions. To this end, we propose PhyParam, a novel framework that integrates object-level forces, mass, friction, and scene gravity into the diffusion process via a lightweight physics-aware attention mechanism, further enhanced by semantic-structural feature supervision to improve dynamic modeling. We introduce PhyParam-Dataset, comprising 130,000 meticulously annotated video clips, and present the first approach enabling explicit physical control over rigid-body motion in image-to-video synthesis. Additionally, we establish PhyParam-Bench, a new evaluation benchmark assessing physical consistency across temporal dynamics, spatial stability, and semantic-physical alignment. Experiments demonstrate that our method significantly enhances physical plausibility while preserving high visual fidelity. Code, dataset, and benchmark are publicly released.

Abstract

Recent advances in image-to-video generation have improved visual realism, making physically grounded and controllable dynamics an important step toward future world simulation. Current models often generate plausible motion, but it is not reliably governed by explicit physical causes, and instance-level constraints can leak or become entangled in multi-object interactions. We attribute this gap to two missing pieces: large-scale, fine-grained physical parameterization, and model designs that correctly bind physical attributes to instances and emphasize dynamics over appearance. To bridge this gap, we introduce PhyParam-Dataset, an interaction-centric collection of 130K physically simulated videos with dense physical parameterization, including force vectors, object material properties, and environmental constants across five representative rigid-body motion types. Built on this data, we present PhyParam, a physics-guided image-to-video diffusion model that conditions on object-level forces, masses, friction, restitution, and scene-level gravity via a lightweight physical-attention routing mechanism, and further improves motion learning with semantic-structural feature-space supervision. We also establish PhyParam-Bench, a benchmark for physical-law consistency in image-to-video generation, with a multi-level protocol evaluating temporal dynamics, spatial stability, and semantic--physical alignment. Experiments show that PhyParam improves physical consistency while maintaining high visual fidelity, advancing explicit rigid-body physical-parameter control for image-to-video generation. We will publicly release the dataset, benchmark, and code to support future research.

Problem

• physical parameter control
• image-to-video generation
• physically grounded dynamics
• multi-object interaction
• physical consistency

Innovation

• physical parameter control
• image-to-video generation
• physics-guided diffusion model
• rigid-body dynamics
• physical consistency benchmark