Targetless Intrinsics and Extrinsic Calibration of Multiple LiDARs and Cameras with IMU using Continuous-Time Estimation

2025-01-06

AI Summary

Calibrating spatiotemporal parameters across asynchronous multi-LiDAR, multi-camera, and IMU systems remains challenging due to hardware-trigger dependency, requirement of calibration targets or overlapping fields-of-view, and sensitivity to scene structure. Method: This paper proposes a target-free, overlap-agnostic continuous-time joint autocalibration framework. It introduces the first targetless continuous-time bundle adjustment formulation, enables cross-modal feature matching between LiDAR intensity images and camera images, and jointly optimizes 6-DoF extrinsics and sensor time delays by integrating Structure-from-Motion, adaptive voxel mapping, and nonlinear optimization (Ceres). Contribution/Results: The method eliminates reliance on hardware synchronization and structured environments, supporting arbitrary numbers of heterogeneous sensors. Evaluated on real-world structured scenes, it achieves sub-0.8-pixel image reprojection error and centimeter-level point-cloud registration accuracy. Calibration results satisfy motion consistency and geometric constraints without drift or cumulative error.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

Accurate spatiotemporal calibration is a prerequisite for multisensor fusion. However, sensors are typically asynchronous, and there is no overlap between the fields of view of cameras and LiDARs, posing challenges for intrinsic and extrinsic parameter calibration. To address this, we propose a calibration pipeline based on continuous-time and bundle adjustment (BA) capable of simultaneous intrinsic and extrinsic calibration (6 DOF transformation and time offset). We do not require overlapping fields of view or any calibration board. Firstly, we establish data associations between cameras using Structure from Motion (SFM) and perform self-calibration of camera intrinsics. Then, we establish data associations between LiDARs through adaptive voxel map construction, optimizing for extrinsic calibration within the map. Finally, by matching features between the intensity projection of LiDAR maps and camera images, we conduct joint optimization for intrinsic and extrinsic parameters. This pipeline functions in texture-rich structured environments, allowing simultaneous calibration of any number of cameras and LiDARs without the need for intricate sensor synchronization triggers. Experimental results demonstrate our method's ability to fulfill co-visibility and motion constraints between sensors without accumulating errors.

Problem

• Multi-sensor Calibration
• Time-space Alignment
• Target-less Calibration

Innovation

• Multi-sensor Calibration
• Self-alignment
• Texture-rich Environment