VideoPhysEdit: Physical Counterfactual Video Editing via Rigid-Body Physical Scene Reconstruction

2026-09-28

AI Summary

This study addresses the limitation of existing video editing models in inferring subsequent motions arising from physical interactions. To this end, we propose the first physics-based counterfactual editing paradigm and a training-free pipeline that explicitly reasons about physical laws through rigid-body scene reconstruction and physics simulation, integrated with guided generation to produce physically plausible counterfactual video edits. Furthermore, this work constructs the PCVE-RigidBench benchmark and introduces the Physical Edit Score as an evaluation metric. Experimental results demonstrate that the proposed method significantly outperforms mainstream open-source and commercial models on both real-world and synthetic data, emerging as the only approach to achieve a positive physical editing score.

Abstract

Video editing has advanced substantially in recent years, with methods increasingly accounting for the visual consequences of edits, such as changes to shadows and occlusions. However, the physical consequences of edits, including changes to subsequent motion and interactions, remain less explored. We formulate this problem as physical counterfactual video editing (PCVE), which aims to generate a counterfactual video depicting the resulting motion and interactions given a source video, a physical edit, and its execution frame. PCVE is challenging because it requires understanding scene physics and inferring the downstream motion and interactions induced by a physical intervention, while paired factual and counterfactual data and dedicated evaluation metrics are lacking. We introduce VideoPhysEdit, a new training-free pipeline for PCVE in rigid-body scenes. It makes physical reasoning explicit through a novel physical scene reconstruction method that recovers a scene reproducing the observed motion and interactions under simulation, enabling the pipeline to apply physical edits as interventions and use the resulting trajectories to guide counterfactual video generation. We further construct PCVE-RigidBench, a synthetic benchmark with paired source and counterfactual target videos and physical ground truth, and introduce the Physical Edit Score. VideoPhysEdit achieves substantially higher physical edit accuracy than open-source methods and commercial models while maintaining competitive visual fidelity. Its Physical Edit Score is 0.376, the only positive score among the compared methods. Qualitative comparisons on real videos further show that VideoPhysEdit applies to real-world scenes and better depicts the downstream motion and interactions induced by the edits than the compared methods. Code: https://github.com/Hammour-steak/VideoPhysEdit

Problem

• Physical counterfactual video editing
• Video editing
• Rigid-body physics
• Physical scene reconstruction

Innovation

• Physical Counterfactual Video Editing
• Rigid-Body Scene Reconstruction
• Training-free Pipeline
• Physics-based Reasoning
• Evaluation Benchmark