Degradation-Aware Cooperative Multi-Modal GNSS-Denied Localization Leveraging LiDAR-Based Robot Detections

2025-10-23

AI Summary

In GNSS-denied environments, collaborative multi-robot localization faces challenges in fusing asynchronous, heterogeneous, multimodal sensor data—such as visual-inertial odometry (VIO), LiDAR-inertial odometry (LIO), and cross-robot 3D detection—while remaining vulnerable to sensor degeneracy. To address this, we propose a degeneracy-aware, loosely coupled factor graph framework for heterogeneous multimodal fusion. Our method innovatively employs the Wasserstein distance to dynamically weight VIO measurements and quantifies LIO degeneracy via the Hessian matrix of scan matching. Additionally, we introduce interpolation factors to unify asynchronous data streams within the factor graph. Evaluated on real-world UGV–UAV heterogeneous platform datasets, our approach significantly improves long-term localization accuracy and robustness under severe degeneracy conditions—including low-texture scenes, motion blur, and sparse geometric structures—demonstrating strong practicality for real-world deployment.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Accurate long-term localization using onboard sensors is crucial for robots operating in Global Navigation Satellite System (GNSS)-denied environments. While complementary sensors mitigate individual degradations, carrying all the available sensor types on a single robot significantly increases the size, weight, and power demands. Distributing sensors across multiple robots enhances the deployability but introduces challenges in fusing asynchronous, multi-modal data from independently moving platforms. We propose a novel adaptive multi-modal multi-robot cooperative localization approach using a factor-graph formulation to fuse asynchronous Visual-Inertial Odometry (VIO), LiDAR-Inertial Odometry (LIO), and 3D inter-robot detections from distinct robots in a loosely-coupled fashion. The approach adapts to changing conditions, leveraging reliable data to assist robots affected by sensory degradations. A novel interpolation-based factor enables fusion of the unsynchronized measurements. LIO degradations are evaluated based on the approximate scan-matching Hessian. A novel approach of weighting odometry data proportionally to the Wasserstein distance between the consecutive VIO outputs is proposed. A theoretical analysis is provided, investigating the cooperative localization problem under various conditions, mainly in the presence of sensory degradations. The proposed method has been extensively evaluated on real-world data gathered with heterogeneous teams of an Unmanned Ground Vehicle (UGV) and Unmanned Aerial Vehicles (UAVs), showing that the approach provides significant improvements in localization accuracy in the presence of various sensory degradations.

Problem

• Achieving accurate multi-robot localization in GNSS-denied environments
• Fusing asynchronous multi-modal sensor data from moving robots
• Adapting to sensory degradations to improve localization robustness

Innovation

• Adaptive factor-graph fuses multi-modal robot data cooperatively
• Interpolation factor enables fusion of unsynchronized sensor measurements
• Wasserstein distance weights VIO data for degradation adaptation