Joint Multi-Camera LiDAR Extrinsic Calibration via Learned Pairwise Initialization and Geometric Refinement

2026-05-29

AI Summary

This work addresses the system-level inconsistency in extrinsic calibration of multi-camera–LiDAR systems caused by independent estimation for each camera. To resolve this, the authors propose a two-stage joint calibration framework: first, the CMRNext network is employed to obtain initial extrinsics and 2D–3D correspondences for each camera–LiDAR pair; subsequently, a multi-frame bundle adjustment jointly optimizes all extrinsics by integrating reprojection errors with single-camera priors and inter-camera relative pose constraints, yielding globally consistent estimates. This approach uniquely combines learning-based pairwise initialization with explicit multi-camera geometric constraints. Evaluated on KITTI, it achieves a translation error of 0.89 cm and a rotation error of 0.038°, and on the Walkley dataset, it reduces translation error from 108.6 cm to 3.1 cm, significantly enhancing cross-domain robustness and calibration accuracy.

Abstract

Most learning-based camera-LiDAR calibration methods treat each camera-LiDAR pair independently, ignoring the rigid geometric coupling in multi-camera platforms. As a result, per-camera estimates may be individually accurate yet inconsistent at the system level. We present a two-stage framework for joint multi-camera LiDAR extrinsic calibration that combines learned pairwise matching with geometric refinement. First, CMRNext is applied independently to each camera to produce initial extrinsic estimates and dense 2D-3D correspondences. These predictions are then jointly refined through a multi-frame bundle adjustment with reprojection, per-camera prior, and relative-pose prior terms. This approach converts pairwise predictions into a globally consistent multi-camera calibration. Experiments on KITTI (in-domain for CMRNext) and Walkley (out-of-domain) datasets show improved per-camera accuracy and inter-camera consistency. On KITTI, the method achieves 0.89 cm translation error and 0.038 rotation error. On Walkley, it reduces translation error from 108.6 cm to 3.1 cm, highlighting the benefit of explicit multi-camera coupling when single-camera predictions are less reliable.

Problem

• multi-camera LiDAR calibration
• extrinsic calibration
• geometric consistency
• sensor fusion
• rigid coupling

Innovation

• multi-camera LiDAR calibration
• learned pairwise initialization
• geometric refinement
• bundle adjustment
• global consistency