Data Scaling Laws for End-to-End Autonomous Driving

2025-04-06

AI Summary

This work addresses the data efficiency challenge in end-to-end autonomous driving models by systematically characterizing performance scaling with training dataset size. Using 16–8,192 hours of real-world driving data, we employ a differentiable end-to-end architecture with multi-task joint training and evaluate performance via both open-loop motion prediction metrics and closed-loop CARLA simulation. We establish, for the first time, the data scaling law for such models: key metrics—particularly trajectory prediction accuracy—exhibit power-law growth with respect to data volume; achieving a 5% relative accuracy gain requires approximately a fourfold increase in training data. Our analysis quantifies the data investment required to attain target performance improvements, providing the first reproducible, quantitative foundation for data-centric autonomous vehicle development. This advances the paradigm shift from “model-centric” design toward “data–model co-optimization” in AV research and engineering.

Technology Category

• Machine Learning: Scalability of ML Systems
• Computer Vision: Large Vision Models
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Responsible Web: Ethical and legal aspects of web-scale data analysis, uses and collection practices
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success

Abstract

Autonomous vehicle (AV) stacks have traditionally relied on decomposed approaches, with separate modules handling perception, prediction, and planning. However, this design introduces information loss during inter-module communication, increases computational overhead, and can lead to compounding errors. To address these challenges, recent works have proposed architectures that integrate all components into an end-to-end differentiable model, enabling holistic system optimization. This shift emphasizes data engineering over software integration, offering the potential to enhance system performance by simply scaling up training resources. In this work, we evaluate the performance of a simple end-to-end driving architecture on internal driving datasets ranging in size from 16 to 8192 hours with both open-loop metrics and closed-loop simulations. Specifically, we investigate how much additional training data is needed to achieve a target performance gain, e.g., a 5% improvement in motion prediction accuracy. By understanding the relationship between model performance and training dataset size, we aim to provide insights for data-driven decision-making in autonomous driving development.

Problem

• Evaluates end-to-end AV performance with varying dataset sizes
• Investigates data needed for target performance gains in AV
• Analyzes relationship between model performance and training data

Innovation

• End-to-end differentiable model integration
• Data scaling for performance optimization
• Closed-loop simulation for evaluation