🤖 AI Summary
High-level autonomous driving (HAD) systems suffer from poor cross-hardware generalization due to multi-modal perception models’ strong dependence on specific sensor hardware configurations.
Method: This paper proposes the first sensor data abstraction framework tailored for HAD, systematically defining and implementing unified abstraction interfaces for cameras, LiDAR, and millimeter-wave radar. Integrating signal processing, geometric modeling, and representation learning, the framework enables hardware-agnostic representations for both uni-modal and multi-modal fusion.
Contribution/Results: Evaluated on diverse real-world datasets, this work identifies— for the first time—the core challenges and technical pathways for abstracting all three sensor modalities. It establishes a theoretical foundation and architectural blueprint for building scalable, generalizable perception models that transcend hardware-specific constraints, thereby advancing robust, deployment-ready HAD systems.
📝 Abstract
Full-stack autonomous driving perception modules usually consist of data-driven models based on multiple sensor modalities. However, these models might be biased to the sensor setup used for data acquisition. This bias can seriously impair the perception models’ transferability to new sensor setups, which continuously occur due to the market's competitive nature. We envision sensor data abstraction as an interface between sensor data and machine learning applications for highly automated vehicles (HAD). For this purpose, we review the primary sensor modalities, camera, lidar, and radar, published in autonomous-driving related datasets, examine single sensor abstraction and abstraction of sensor setups, and identify critical paths towards an abstraction of sensor data from multiple perception configurations.
