ShotPlan: Cinematic Video Generation with Learnable Planning Token

2026-07-20

AI Summary

Current video generation models excel at single-shot synthesis but still struggle with weak narrative coherence and inflexible shot composition in cinematic multi-shot video generation. This work proposes a novel approach based on a video diffusion foundation model, introducing learnable planning tokens to explicitly model shot structure and integrating Fractional Rotary Position Embedding (FRoPE) to enable frame-accurate control over shot transitions. The method achieves, for the first time, precise temporal modeling of multi-shot sequences, significantly enhancing inter-shot consistency and scheduling flexibility while preserving high visual fidelity. It demonstrates clear superiority over existing methods on cinematic video generation benchmarks.

Abstract

Current video generation models achieve impressive results in single-shot generation, yet remain limited in cinematic video generation, where coherent narratives and effective multi-shot composition require explicit shot planning. To address this challenge, we propose ShotPlan, a framework for explicit multi-shot cinematic video generation built upon a video diffusion foundation model. Our method introduces learnable planning tokens that capture shot-level transition cues and can be seamlessly integrated with the original video generation tokens to control transition timestamps. Unlike standard video generation tokens, the proposed planning tokens are equipped with Fractional Temporal Rotary Position Embedding (FRoPE), enabling shot transitions to be modeled at the frame level. Experiments demonstrate that ShotPlan significantly outperforms existing cinematic video generation methods, offering more flexible shot management and stronger inter-shot consistency.

Problem

• cinematic video generation
• multi-shot composition
• shot planning
• coherent narratives
• video generation

Innovation

• planning tokens
• multi-shot video generation
• Fractional Temporal Rotary Position Embedding
• cinematic video generation
• video diffusion model