Imitation Learning for Autonomous Driving: Insights from Real-World Testing

2025-04-26

AI Summary

End-to-end autonomous driving on real-world tracks demands simultaneous real-time performance, robustness to environmental variations, and smooth control—challenging for resource-constrained embedded platforms. Method: We conduct imitation learning on the MIT Racecar platform, proposing an incremental model design paradigm and systematically comparing CNN, CNN-LSTM, and CNN-NODE architectures for temporal modeling and driving robustness in dynamic scenarios. Contribution/Results: This work presents the first validation of CNN-NODE’s superior adaptability to illumination changes and sharp turns on a miniature racing platform. Relative to conventional PD controllers and vanilla CNNs, both CNN-LSTM and CNN-NODE significantly improve steering stability and trajectory smoothness. The deployed system achieves a real-time sampling rate of 30 Hz with end-to-end latency under 40 ms. Our empirical study provides reproducible model selection guidelines and an engineering implementation framework for reliable edge-deployed end-to-end driving under strict computational constraints.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Vision for Robotics & Autonomous Driving
• Planning, Routing, and Scheduling: Model-Based Reasoning

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

This work focuses on the design of a deep learning-based autonomous driving system deployed and tested on the real-world MIT Racecar to assess its effectiveness in driving scenarios. The Deep Neural Network (DNN) translates raw image inputs into real-time steering commands in an end-to-end learning fashion, following the imitation learning framework. The key design challenge is to ensure that DNN predictions are accurate and fast enough, at a high sampling frequency, and result in smooth vehicle operation under different operating conditions. In this study, we design and compare various DNNs, to identify the most effective approach for real-time autonomous driving. In designing the DNNs, we adopted an incremental design approach that involved enhancing the model capacity and dataset to address the challenges of real-world driving scenarios. We designed a PD system, CNN, CNN-LSTM, and CNN-NODE, and evaluated their performance on the real-world MIT Racecar. While the PD system handled basic lane following, it struggled with sharp turns and lighting variations. The CNN improved steering but lacked temporal awareness, which the CNN-LSTM addressed as it resulted in smooth driving performance. The CNN-NODE performed similarly to the CNN-LSTM in handling driving dynamics, yet with slightly better driving performance. The findings of this research highlight the importance of iterative design processes in developing robust DNNs for autonomous driving applications. The experimental video is available at https://www.youtube.com/watch?v=FNNYgU--iaY.

Problem

• Designing DNNs for real-time autonomous driving commands
• Ensuring accurate and smooth vehicle operation in diverse conditions
• Comparing DNN architectures for optimal imitation learning performance

Innovation

• End-to-end DNN for real-time steering commands
• Incremental design enhancing model and dataset
• CNN-LSTM for smooth temporal-aware driving