An LSTM-Based Autonomous Driving Model Using Waymo Open Dataset

2020-02-14

Applied Sciences

AI Summary

This study addresses the opacity of low-level driving policies in the Waymo autonomous driving system. We propose a data-driven, end-to-end inverse modeling approach leveraging publicly available Waymo Open Dataset. Our method fuses time-series multimodal sensor data—including LiDAR, camera, and IMU inputs—to train an LSTM-based model that directly predicts vehicle control actions (e.g., longitudinal acceleration and steering angle), bypassing hand-crafted rule-based assumptions. To our knowledge, this is the first work to perform data-driven inverse modeling of Waymo’s black-box driving policy using its open dataset, significantly improving behavioral fidelity between simulation and real-world deployment. Experimental results demonstrate that our model reduces mean absolute error (MAE) in acceleration prediction by 12.6%–28.3% over multiple baselines. Furthermore, trajectory visualization and behavioral plausibility analysis confirm the model’s physical interpretability and generalization capability across diverse driving scenarios.

Technology Category

• Planning, Routing, and Scheduling: Model-Based Reasoning
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The Waymo Open Dataset has been released recently, providing a platform to crowdsource some fundamental challenges for automated vehicles (AVs), such as 3D detection and tracking. While the dataset provides a large amount of high-quality and multi-source driving information, people in academia are more interested in the underlying driving policy programmed in Waymo self-driving cars, which is inaccessible due to AV manufacturers’ proprietary protection. Accordingly, academic researchers have to make various assumptions to implement AV components in their models or simulations, which may not represent the realistic interactions in real-world traffic. Thus, this paper introduces an approach to learn a long short-term memory (LSTM)-based model for imitating the behavior of Waymo’s self-driving model. The proposed model has been evaluated based on Mean Absolute Error (MAE). The experimental results show that our model outperforms several baseline models in driving action prediction. In addition, a visualization tool is presented for verifying the performance of the model.

Problem

• Achieving human-level accuracy in autonomous driving decisions
• Predicting vehicle acceleration using CNN and LSTM models
• Leveraging Waymo's dataset for self-driving car behavior modeling

Innovation

• Using CNN to mimic human driving actions
• Applying LSTM for time series analysis
• Leveraging Waymo's open dataset for training