CinemaTraj: Composing Atomic Camera Trajectories for 3D Scenes with LLM Agents

2026-07-29

AI Summary

This work addresses the challenge of generating cinematic 3D camera trajectories from natural language by proposing a semantic–geometric joint reasoning framework that overcomes the limitations of existing methods, which often lack realistic spatial awareness or neglect cinematic semantics. The approach models the task as language-guided spatial reasoning, leveraging a large language model (LLM) agent in conjunction with a structured 3D scene graph to decompose user prompts into parameterized, atomic cinematographic camera motions. Concurrently, it generates synchronized narration and subtitles. The framework enables an optimizable, semantically meaningful trajectory representation and incorporates a collision-avoidance mechanism. Evaluated on real-world ScanNet++ scenes, the method produces trajectories that outperform state-of-the-art approaches in prompt alignment, visual quality, and motion safety.

Abstract

Automatically generating cinematically expressive camera trajectories through 3D scenes from natural language descriptions is a challenging task of high practical value, with applications ranging from real-estate advertising to virtual tour creation. Existing methods either lack true 3D spatial awareness by relying on 2D image priors, or treat trajectory generation as a geometric path planning problem divorced from cinematographic semantics. We present CinemaTraj, a framework that reframes camera trajectory planning as a language-grounded spatial reasoning problem. Given a set of RGB-D images and a user prompt, CinemaTraj equips an LLM agent with a structured 3D scene graph: the agent decomposes the prompt into a sequence of atomic cinematographic movements (dolly, orbit, crane, pan, tilt, zoom, arc). Each movement is instantiated via a novel parametric trajectory representation that is both cinematographically expressive and optimizable for collision avoidance. The scene graph acts as a structured spatial prior, grounding the agent's reasoning in accurate geometric and semantic knowledge of the environment. CinemaTraj further generates synchronized voiceover and subtitles aligned with camera motion, producing narrated cinematic video outputs. We evaluate CinemaTraj on real-world ScanNet++ environments, and show that it produces prompt-faithful, collision-free trajectories with high cinematographic quality, outperforming existing approaches on prompt alignment, trajectory quality, and safety metrics.

Problem

• camera trajectory generation
• 3D scene understanding
• cinematographic semantics
• natural language grounding
• spatial reasoning

Innovation

• camera trajectory generation
• LLM agents
• 3D scene graph
• atomic cinematographic movements
• parametric trajectory representation