GRASPTrack: Geometry-Reasoned Association via Segmentation and Projection for Multi-Object Tracking

2025-08-11

AI Summary

Monocular multi-object tracking (MOT) suffers from degraded performance in highly occluded and depth-ambiguous scenes, primarily due to the lack of explicit geometric modeling in conventional detection-driven approaches. To address this, we propose a depth-aware tracking framework: first, high-fidelity 3D point clouds are generated by fusing monocular depth estimation with instance segmentation; second, voxelization enables differentiable 3D IoU computation for robust spatial association; third, we introduce a depth-aware adaptive noise compensation mechanism and a depth-enhanced observation-centric momentum strategy to improve robustness in motion state estimation and ensure 3D motion consistency. Extensive experiments on MOT17, MOT20, and DanceTrack demonstrate significant improvements in tracking accuracy and stability—particularly under severe occlusion and complex motion patterns—outperforming state-of-the-art methods while maintaining computational efficiency.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: State Estimation
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Security and Privacy: Tracking, profiling, and countermeasures against them
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Multi-object tracking (MOT) in monocular videos is fundamentally challenged by occlusions and depth ambiguity, issues that conventional tracking-by-detection (TBD) methods struggle to resolve owing to a lack of geometric awareness. To address these limitations, we introduce GRASPTrack, a novel depth-aware MOT framework that integrates monocular depth estimation and instance segmentation into a standard TBD pipeline to generate high-fidelity 3D point clouds from 2D detections, thereby enabling explicit 3D geometric reasoning. These 3D point clouds are then voxelized to enable a precise and robust Voxel-Based 3D Intersection-over-Union (IoU) for spatial association. To further enhance tracking robustness, our approach incorporates Depth-aware Adaptive Noise Compensation, which dynamically adjusts the Kalman filter process noise based on occlusion severity for more reliable state estimation. Additionally, we propose a Depth-enhanced Observation-Centric Momentum, which extends the motion direction consistency from the image plane into 3D space to improve motion-based association cues, particularly for objects with complex trajectories. Extensive experiments on the MOT17, MOT20, and DanceTrack benchmarks demonstrate that our method achieves competitive performance, significantly improving tracking robustness in complex scenes with frequent occlusions and intricate motion patterns.

Problem

• Addresses occlusion and depth ambiguity in monocular multi-object tracking
• Integrates depth estimation and segmentation for 3D geometric reasoning
• Enhances tracking robustness with adaptive noise and 3D motion cues

Innovation

• Integrates depth estimation and segmentation for 3D reasoning
• Uses voxelized 3D IoU for precise spatial association
• Employs depth-aware noise compensation for reliable tracking