ORB-SfMLearner: ORB-Guided Self-supervised Visual Odometry with Selective Online Adaptation

2024-09-18

arXiv.org

AI Summary

To address the low accuracy and poor generalization of deep visual odometry (VO), this paper proposes an ORB feature-guided self-supervised monocular VO framework. Methodologically, we pioneer the integration of ORB keypoints into self-supervised ego-motion learning to strengthen geometric constraints; design a cross-attention mechanism to explicitly model correlations between motion direction and feature responses, thereby improving pose estimation robustness and interpretability; and introduce a selective online adaptation mechanism enabling parameter-efficient fine-tuning with only a few frames for cross-domain scenarios. Evaluated on KITTI and vKITTI, our method reduces absolute trajectory error by 12.3% over state-of-the-art methods, demonstrating significant improvements in cross-domain transfer capability and real-time adaptability.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Deep visual odometry, despite extensive research, still faces limitations in accuracy and generalizability that prevent its broader application. To address these challenges, we propose an Oriented FAST and Rotated BRIEF (ORB)-guided visual odometry with selective online adaptation named ORB-SfMLearner. We present a novel use of ORB features for learning-based ego-motion estimation, leading to more robust and accurate results. We also introduce the cross-attention mechanism to enhance the explainability of PoseNet and have revealed that driving direction of the vehicle can be explained through the attention weights. To improve generalizability, our selective online adaptation allows the network to rapidly and selectively adjust to the optimal parameters across different domains. Experimental results on KITTI and vKITTI datasets show that our method outperforms previous state-of-the-art deep visual odometry methods in terms of ego-motion accuracy and generalizability.

Problem

• Deep Visual Odometry
• Accuracy Improvement
• Environmental Adaptability

Innovation

• ORB-SfMLearner
• Cross-Attention Mechanism
• Deep Learning Integration