Bridge 2D-3D: Uncertainty-aware Hierarchical Registration Network with Domain Alignment

2025-04-02

AI Summary

To address cross-modal matching bias and domain discrepancy in image-to-point-cloud rigid registration, this paper proposes a coarse-to-fine uncertainty-aware registration framework. Methodologically, it integrates multi-level feature interaction, end-to-end differentiable registration, and adversarial training. Key contributions include: (1) an Uncertainty-aware Hierarchical Matching Module (UHMM) that models feature matching confidence to enable selective fusion of salient patches, thereby suppressing noise; and (2) an Adversarial Modality Alignment Module (AMAM) that jointly optimizes cross-modal feature distribution alignment and geometric consistency to substantially narrow the domain gap. Evaluated on RGB-D Scene V2 and 7-Scenes benchmarks, the method achieves state-of-the-art performance, with significant improvements in registration accuracy and enhanced robustness against sensor noise and occlusion.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

The method for image-to-point cloud registration typically determines the rigid transformation using a coarse-to-fine pipeline. However, directly and uniformly matching image patches with point cloud patches may lead to focusing on incorrect noise patches during matching while ignoring key ones. Moreover, due to the significant differences between image and point cloud modalities, it may be challenging to bridge the domain gap without specific improvements in design. To address the above issues, we innovatively propose the Uncertainty-aware Hierarchical Matching Module (UHMM) and the Adversarial Modal Alignment Module (AMAM). Within the UHMM, we model the uncertainty of critical information in image patches and facilitate multi-level fusion interactions between image and point cloud features. In the AMAM, we design an adversarial approach to reduce the domain gap between image and point cloud. Extensive experiments and ablation studies on RGB-D Scene V2 and 7-Scenes benchmarks demonstrate the superiority of our method, making it a state-of-the-art approach for image-to-point cloud registration tasks.

Problem

• Addresses uncertainty in matching image patches to point clouds
• Reduces domain gap between image and point cloud modalities
• Improves hierarchical registration accuracy via multi-level feature fusion

Innovation

• Uncertainty-aware Hierarchical Matching Module
• Adversarial Modal Alignment Module
• Multi-level fusion interactions