HelloWorld: Towards Practical Applications of Generative Driving World Models

2026-09-23

AI Summary

This study addresses the challenges of scene generalization, multi-sensor consistency, and inference efficiency in autonomous driving world models by proposing a 2B-parameter driving world model. Methodologically, it leverages heterogeneous video pretraining to acquire visual motion priors and performs conditional generation incorporating ego-vehicle pose, high-definition maps, and 3D bounding boxes. Furthermore, a block-causal generation interface and an adaptive context mechanism are introduced to enable the synchronous generation of seven-camera RGB and LiDAR data alongside few-step inference acceleration. This work significantly improves generated visual quality, control fidelity, and cross-view consistency while enhancing robustness across repeated generations, thereby establishing a new paradigm for efficient, controllable multi-view simulation and data synthesis.

Abstract

Driving world models provide a promising route toward scalable counterfactual data generation and interactive simulation beyond recorded driving logs. Realizing this potential requires a system that can generalize across diverse scenes, respond faithfully to prescribed controls, generate coherent multi-sensor observations, and operate efficiently under repeated inference. We present \textbf{HelloWorld}, a 2B driving world model system designed around these requirements. HelloWorld progressively specializes broad visual and motion priors from heterogeneous video data into controllable driving generation using ego pose, HD maps, and 3D boxes. A block-causal generation interface, together with adaptation to self-generated context, aligns the model with sequential simulation. The system further supports synchronized seven-camera RGB generation and conditional LiDAR synthesis, and is distilled toward few-step inference for efficient deployment. Experiments evaluate visual quality, control fidelity, cross-view consistency, robustness under repeated generation, inference efficiency, and LiDAR synthesis. Together, HelloWorld provides a unified framework for scalable driving data generation and interactive simulation.

Problem

• Driving World Models
• Counterfactual Data Generation
• Interactive Simulation
• Multi-sensor Generation
• Scalable Driving

Innovation

• Driving World Models
• Controllable Generation
• Multi-sensor Synthesis
• Block-causal Generation
• Model Distillation