Pictura: Perspective-View Self-Play at Scale for Driving

2026-07-28

AI Summary

This work addresses the representational gap that arises when driving policies trained with privileged state information are deployed using only first-person visual inputs. To bridge this gap, the authors propose the first purely vision-based self-play training framework that learns driving policies end-to-end directly from agent-centric images. Leveraging the GPU-accelerated multi-agent simulator Pictura and the PPO algorithm, the method achieves highly efficient training—processing 500,000 agent steps (equivalent to 2 million images) per second on a single H100 GPU. The resulting policy, Alberti, trained over 50 billion agent steps (approximately 35 million kilometers), closely matches the performance of privileged-observation baselines and demonstrates zero-shot superiority on re-rendered Waymo scenarios, effectively closing the perception gap between simulation and real-world deployment.

Abstract

Self-play in simulation produces robust driving policies at scale. Demonstrations of such behavior have been made using privileged vectorized observations such as exact poses and velocities, even for occluded agents. This assumes that perception is solved and introduces a representation gap with the partial observation of a deployed agent driving from the perspective view of egocentric cameras. A common fix, distilling the privileged policy into a camera-input student, leaves the student imitating decisions its own view cannot justify. Instead, we establish perspective-view self-play as a practical training regime. We introduce Pictura, a GPU-accelerated multi-agent driving simulator that renders each agent's egocentric view at every step, mitigating the representation gap at its source. Pictura sustains up to 500K agent-steps/s (2M images/s) on a single H100. Using Pictura, we train Alberti by self-play with plain PPO. It is the first large-scale driving self-play policy trained directly from perspective images, without privileged observations. Training spans 50B agent steps for ~35M km of driving. It approaches the driving performance of its privileged vectorized counterpart, and transfers zero-shot to Waymo Open Motion Dataset layouts re-rendered in Pictura, where it outperforms privileged vectorized agents. Project page: https://valeoai.github.io/Pictura/

Problem

• self-play
• driving policy
• perspective view
• representation gap
• privileged observation

Innovation

• perspective-view self-play
• GPU-accelerated simulation
• privileged observation-free
• egocentric vision-based driving
• large-scale reinforcement learning