Problem
Research questions and friction points this paper is trying to address.
Referring Multi-Object Tracking
Moving-Camera Videos
Camera Motion
Motion Compensation
Residual Motion
Referring Multi-Object Tracking in Moving-Camera Videos via Global Motion Compensation
🤖 AI Summary
This study addresses the challenge in moving-camera videos where camera motion causes discrepancies between true object displacement and visual appearance, hindering existing methods from accurately matching motion attributes described in natural language. To overcome this, we propose a residual motion-based semantic matching module that explicitly eliminates camera-induced spurious motion through global motion estimation. By extracting inter-frame residuals to characterize genuine object motion dynamics and incorporating a late fusion strategy, the module optimizes multi-object tracking results. Functioning as a plug-and-play component, it significantly enhances tracking accuracy across multiple Referring Multi-Object Tracking (RMOT) baseline models, effectively resolving motion ambiguity in dynamic scenes.
📝 Abstract
Referring multi-object tracking (RMOT) takes a video and a language expression as input and tracks all referred objects. Many tracking requirements involve how an object moves rather than how it appears. However, in a video captured by a moving camera, a parked vehicle may appear to move, while a moving vehicle may show little displacement. Existing RMOT methods relate motion with text but do not explicitly remove camera-induced motion. In this paper, we propose extracting residual motion across frames by estimating camera motion in driver-view videos and compare motion characteristics with the query expression. We consider the motion-matching extent and integrate it with the RMOT method's prediction result through late fusion. In the evaluation, we verify the performance gain of taking the motion compensation module as a plug-in across different RMOT hosts.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Referring Multi-Object Tracking
Global Motion Compensation
Residual Motion Extraction
Late Fusion
Plug-and-Play
🔎 Similar Papers
No similar papers found.
