Technology Category
Computer Vision: Multi-modal VisionMachine Learning: Calibration & Uncertainty QuantificationIntelligent Robots: Multimodal Perception & Sensor Fusion
Cal or No Cal? -- Real-Time Miscalibration Detection of LiDAR and Camera Sensors
🤖 AI Summary
Real-time detection of LiDAR–camera extrinsic parameter misalignment remains challenging in autonomous driving systems. Method: This paper proposes a lightweight end-to-end binary classification framework that reframes conventional parameter regression as a calibration-state discrimination task. We innovatively introduce contrastive learning into multimodal sensor calibration verification, designing a Siamese-network-based cross-modal feature embedding model with a lightweight CNN backbone and cosine similarity–based decision mechanism—eliminating reliance on geometric priors, specific object classes, or driving behavior. Contribution/Results: The method achieves >98% detection accuracy on both the KITTI benchmark and a custom dataset, with inference latency under 10 ms, enabling deployment on embedded platforms. It significantly outperforms state-of-the-art approaches and the source code is publicly available.
Technology Category
Application Category
📝 Abstract
The goal of extrinsic calibration is the alignment of sensor data to ensure an accurate representation of the surroundings and enable sensor fusion applications. From a safety perspective, sensor calibration is a key enabler of autonomous driving. In the current state of the art, a trend from target-based offline calibration towards targetless online calibration can be observed. However, online calibration is subject to strict real-time and resource constraints which are not met by state-of-the-art methods. This is mainly due to the high number of parameters to estimate, the reliance on geometric features, or the dependence on specific vehicle maneuvers. To meet these requirements and ensure the vehicle's safety at any time, we propose a miscalibration detection framework that shifts the focus from the direct regression of calibration parameters to a binary classification of the calibration state, i.e., calibrated or miscalibrated. Therefore, we propose a contrastive learning approach that compares embedded features in a latent space to classify the calibration state of two different sensor modalities. Moreover, we provide a comprehensive analysis of the feature embeddings and challenging calibration errors that highlight the performance of our approach. As a result, our method outperforms the current state-of-the-art in terms of detection performance, inference time, and resource demand. The code is open source and available on https://github.com/TUMFTM/MiscalibrationDetection.
Problem
Research questions and friction points this paper is trying to address.
Detects LiDAR-camera miscalibration in real-time
Uses contrastive learning for binary calibration classification
Addresses limitations of current online calibration methods
Innovation
Methods, ideas, or system contributions that make the work stand out.
Binary classification for calibration state detection
Contrastive learning in latent feature space
Real-time miscalibration detection outperforms existing methods
🔎 Similar Papers
No similar papers found.
