Problem
Research questions and friction points this paper is trying to address.
multi-object tracking
memory drift
re-identification
LiAM-SAM: Lifecycle-Aware Memory for Robust SAM2-Based MOT
🤖 AI Summary
该研究针对基于SAM2的多目标跟踪在复杂场景下的脆弱性问题，提出LiAM-SAM框架，通过生命周期意识记忆机制解决初始化错误、交互时记忆漂移及长时间遮挡后重识别不可靠等问题。
📝 Abstract
Segmentation-based multi-object tracking (MOT) with foundation video models such as SAM2 offers strong localization quality, yet remains fragile in crowded, real-world scenes. In detector-prompted SAM2 pipelines, failures typically arise at three stages of the object lifecycle: (i) erroneous or duplicate track initiation, (ii) memory drift during close interactions, and (iii) unreliable re-identification after long occlusions or re-entry. These errors corrupt object memory and accumulate over time, making long-horizon tracking unstable. In this paper, we reframe MOT as a lifecycle memory integrity problem. We present LiAM-SAM, a Lifecycle-Aware Memory (LiAM) framework with targeted mechanisms for each of the three failure modes. At track birth, to prevent faulty or duplicate initiations, we apply contrastive track initiation, which conditions each prompt on existing nearby tracked instances. To preserve memory integrity during strong interactions, we introduce motion- and geometry-grounded memory correction that resolves interaction confusions and suppresses drift. For reliable re-identification after disappearance, we maintain an adaptive context memory that promotes diverse and trustworthy references as long-term identity anchors. Finally, similarity aware spatial pruning optionally selects the memory tokens to retain at cross-attention time, improving efficiency with minimal accuracy loss. LiAM-SAM represents a modular, detector-agnostic, SAM2-based MOT system that achieves state-of-the-art HOTA and IDF1 on the evaluated benchmarks. In association-challenging environments, our ablations show that LiAM improves a detector+SAM2 baseline by +10.5 HOTA, +17.4 AssA, and reduces identity switches by 96%.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Lifecycle-Aware Memory
contrastive track initiation
memory correction
adaptive context memory
similarity aware spatial pruning
🔎 Similar Papers
No similar papers found.
