Multi-LVI-SAM: A Robust LiDAR-Visual-Inertial Odometry for Multiple Fisheye Cameras

2025-09-06

AI Summary

To address low visual information fusion efficiency and poor cross-camera feature consistency in tightly coupled odometry integrating multiple fisheye cameras, LiDAR, and IMU, this paper proposes a tightly coupled SLAM framework based on panoramic visual feature modeling. Our method introduces: (1) a unified panoramic feature model that maps multi-fisheye observations onto a conformal spherical manifold for cross-view feature normalization; (2) an online extrinsic parameter compensation mechanism to mitigate triangulation inconsistency caused by calibration errors; and (3) joint embedding of panoramic geometric constraints, multi-view reprojection residuals, and IMU preintegration terms within a factor graph optimization framework. Extensive experiments on EuRoC, KITTI, and a custom dataset demonstrate that our system achieves an average 12.7% improvement in localization accuracy over state-of-the-art methods, while exhibiting significantly enhanced robustness against motion blur and illumination variations.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

We propose a multi-camera LiDAR-visual-inertial odometry framework, Multi-LVI-SAM, which fuses data from multiple fisheye cameras, LiDAR and inertial sensors for highly accurate and robust state estimation. To enable efficient and consistent integration of visual information from multiple fisheye cameras, we introduce a panoramic visual feature model that unifies multi-camera observations into a single representation. The panoramic model serves as a global geometric optimization framework that consolidates multi-view constraints, enabling seamless loop closure and global pose optimization, while simplifying system design by avoiding redundant handling of individual cameras. To address the triangulation inconsistency caused by the misalignment between each camera's frame and the panoramic model's frame, we propose an extrinsic compensation method. This method improves feature consistency across views and significantly reduces triangulation and optimization errors, leading to more accurate pose estimation. We integrate the panoramic visual feature model into a tightly coupled LiDAR-visual-inertial system based on a factor graph. Extensive experiments on public datasets demonstrate that the panoramic visual feature model enhances the quality and consistency of multi-camera constraints, resulting in higher accuracy and robustness than existing multi-camera LiDAR-visual-inertial systems.

Problem

• Fusing multi-fisheye camera, LiDAR, inertial data for robust odometry
• Unifying multi-camera observations into single panoramic representation
• Addressing triangulation inconsistency via extrinsic compensation method

Innovation

• Panoramic visual feature model unifying multi-camera observations
• Extrinsic compensation method reducing triangulation errors
• Tightly coupled LiDAR-visual-inertial factor graph integration