SimWAM: A Simple World Action Model for End-to-End Autonomous Driving

2026-08-07

AI Summary

This work addresses the inefficiency of existing World Action Models (WAMs), which rely on computationally expensive future-frame generation during inference, hindering practical deployment. To overcome this limitation, the authors propose a joint flow-matching framework that co-trains a pretrained video expert with a lightweight action expert, using video generation solely as a training signal while entirely discarding the video branch at inference time to enable end-to-end trajectory prediction. An isolated attention mechanism ensures action prediction operates without access to future frames, facilitating flexible substitution of video backbones and independent scaling of the action module. Furthermore, reinforcement learning is integrated to optimize composite driving rewards. Evaluated on NAVSIM, the method achieves a PDMS score of 91.5, significantly outperforming current WAM approaches with lower latency, and demonstrates successful zero-shot transfer to the nuScenes dataset.

Abstract

World-Action Models (WAMs) improve end-to-end autonomous driving by transferring video dynamics priors to action prediction, but existing methods require costly future generation at inference. We present SimWAM, a simple yet effective WAM that uses video generation purely as a training signal. It co-trains a pretrained video expert and a lightweight action expert with joint flow matching. An isolated attention mask keeps action prediction independent of future frames, allowing the video branch to be discarded after training and leaving a self-contained planner that directly predicts trajectories. Since the two experts share no parameters and interact only through a unified attention interface, the video backbone could be replaced and the action expert scaled independently without modifying the learning objective or inference pipeline. We further apply reinforcement learning to optimize a compositional driving reward beyond trajectory imitation. Our SimWAM achieves $91.5$ PDMS on NAVSIM, surpasses state-of-the-art WAM-based planners with substantially lower latency, and transfers zero-shot to nuScenes. These results position SimWAM as a simple yet solid baseline that could readily benefit from advances in video generation for efficient autonomous driving. The code and model weights are available at https://github.com/H-EmbodVis/SimWAM/

Problem

• World-Action Models
• end-to-end autonomous driving
• future video generation
• inference latency
• trajectory prediction

Innovation

• World-Action Model
• End-to-End Autonomous Driving
• Video Generation as Training Signal
• Joint Flow Matching
• Zero-Shot Transfer