CoDrive: Cross-Vehicle World-Consistent Video Generation with Precise Trajectory Control for Cooperative Driving

2026-09-28

AI Summary

This study addresses the challenge of maintaining dynamic scene consistency across multi-vehicle perspectives in existing driving world models by proposing a cross-vehicle multi-view video generation framework. Methodologically, it introduces a novel local-global self-attention mechanism integrated with projected relative positional encoding to model cross-vehicle consistency, alongside shared-coordinate trajectory control and a progressive mixed-task training strategy. Furthermore, a dedicated benchmark dataset, CoDrive-Bench, is released for evaluation. Experimental results demonstrate that the proposed framework preserves high visual quality while significantly enhancing trajectory controllability as well as geometric and instance-level consistency in multi-agent cooperative driving scenarios.

Abstract

Real-world driving is inherently multi-agent, yet most existing driving world models generate observations from a single ego vehicle. Independently extending them to multiple vehicles does not ensure that different agents observe a consistent shared world. We present CoDrive, a cross-vehicle, multi-view driving video generation framework that jointly generates observations of vehicles sharing the same dynamic scene with precise camera-trajectory control. CoDrive interleaves local self-attention, which models spatiotemporal dependencies among the views of each vehicle, with global self-attention, which enables information exchange and consistency modeling across vehicles. To explicitly encode their spatial relationships, all camera trajectories are represented in a shared world coordinate system and injected into the attention layers through projective relative positional encoding. We further adopt a progressive mixed-task training strategy that combines large-scale real-world single-agent data with synthetic cross-agent interaction data, allowing the model to benefit from real-world appearance distributions while learning cross-agent consistency from simulation. For systematic evaluation, we introduce CoDrive-Bench, a benchmark covering real and synthetic multi-vehicle scenarios and evaluating trajectory controllability, scene geometry consistency, and instance-level consistency. Experiments show that CoDrive improves trajectory controllability and cross-agent geometric and instance consistency while maintaining competitive visual quality.

Problem

• driving world model
• multi-agent
• cross-vehicle video generation
• world consistency
• trajectory control

Innovation

• Cross-vehicle video generation
• World consistency
• Trajectory control
• Progressive mixed-task training
• Multi-agent driving