4Director: Controlling Video World Models with Rigid 3D Geometry

2026-10-01

AI Summary

This study addresses the limitations of existing video generation methods, which lack precise control over camera and object motion while exhibiting insufficient geometric consistency. To this end, this work proposes a video world model grounded in an explicit 4D scene representation. The model introduces geometric constraints through rigid 3D mesh reconstruction to suppress artifacts in unobserved regions, and integrates depth-aware video rendering with a motion adapter to achieve precise motion control. Furthermore, an automatic annotation pipeline, the RealCOD-Rigid dataset, and the IG-IoU evaluation metric are constructed to support this research. Experimental results demonstrate that the proposed method significantly outperforms existing baselines in both visual quality and the precision of camera and object motion control.

Abstract

Precise control over camera and object motion is essential for professional video production. Existing methods control objects only coarsely, through image-plane cues that are ambiguous in depth and rotation or through 3D tracks and blobs that lack complete geometry and lose consistency across viewpoint changes. We introduce 4Director, a video world model conditioned on an explicit 4D scene representation: each object is reconstructed once from the input image as a canonical mesh and moved by one prescribed rigid transformation per frame. This representation provides an intuitive 3D control interface and prevents unobserved geometry from being regenerated independently in every frame. We render the controlled scene as a depth video and introduce a Motion Adapter that transforms this geometric scaffold into video while synthesizing view-consistent appearance, illumination, and non-rigid dynamics. For training, we construct RealCOD-Rigid, a new dataset of 20,774 clips annotated with rigid 3D scenes by our automatic pipeline. We further introduce Identity-Gated IoU (IG-IoU), which jointly evaluates adherence to prescribed object motion and preservation of object identity. Experiments demonstrate that 4Director consistently outperforms prior methods in visual quality and in camera and object control.

Problem

• video world models
• camera and object control
• rigid 3D geometry
• 4D scene representation

Innovation

• Video World Model
• 4D Scene Representation
• Rigid 3D Geometry
• Motion Adapter
• IG-IoU