Robust 6-DoF Object Pose Tracking with Built-In Recovery under Occlusions and Rapid Object Motions

2026-07-26

AI Summary

This work addresses the challenge of robust 6-degree-of-freedom object pose tracking in RGB-D data under severe occlusion and rapid motion, where existing methods often fail and struggle to recover autonomously. To this end, we propose a novel tracking framework that integrates learned keypoint matching with optimization-driven alignment. A reliability monitoring mechanism is introduced to detect tracking failure in real time and trigger a global re-detection and verification process, enabling fully automatic re-initialization without human intervention. Evaluated on both standard and newly constructed challenging datasets, our method achieves state-of-the-art accuracy at 57.6 FPS and demonstrates significantly superior robustness and sustained tracking capability in scenarios involving heavy occlusion and high-speed motion.

Abstract

Real-time 6-DoF object pose tracking is essential for many robotics applications, and several approaches exist. Yet even today's approaches remain unreliable under temporary full occlusions and rapid object motions. Once tracking is lost, most methods struggle to detect the failure and recover automatically, often requiring manual re-initialization. In this paper, we address the problem of robust model-based 6-DoF tracking of unseen objects from RGB-D data, especially in scenarios with occlusion and fast motion. We propose a novel method that combines efficient learning-based keypoint matching with optimization-based alignment and introduces a novel failure detection and recovery module. Our system monitors pose reliability, detects tracking divergence or occlusions, and performs a global re-detection and pose estimation step that robustly verifies recovery candidates before resuming tracking. Our evaluation on standard tracking benchmarks and on a new dataset of occluded and fast-moving scenes shows that our method matches state-of-the-art accuracy on easy tracking sequences, maintains high tracking speed at 57.6 frames per second, and provides the most robust tracking performance under challenging conditions. Thus, we believe that our approach is a relevant step forward in robust 6-DoF object tracking from RGB-D data.

Problem

• 6-DoF object pose tracking
• occlusion
• rapid motion
• failure recovery
• RGB-D data

Innovation

• 6-DoF object pose tracking
• failure detection and recovery
• RGB-D tracking
• occlusion robustness
• real-time pose estimation