Improving Accuracy and Generalization for Efficient Visual Tracking

2024-11-28

arXiv.org

AI Summary

Existing efficient visual trackers achieve strong in-distribution (ID) performance but suffer from poor generalization and low robustness under out-of-distribution (OOD) conditions, hindering deployment in resource-constrained real-world scenarios. To address this, we propose SiamABC—a lightweight Siamese tracker jointly optimized for high ID accuracy and strong OOD generalization. Its key contributions are: (1) the first backpropagation-free test-time adaptation (TTA) mechanism enabling rapid dynamic self-adaptation during inference; (2) a target-dynamic modeling architecture integrated with a lightweight feature bridging module; and (3) a customized contrastive-consistency joint loss function. On the OOD AVisT benchmark, SiamABC outperforms MixFormerV2-S by 7.6% while achieving 100 FPS on CPU—three times faster—without compromising leading ID performance.

Technology Category

• Computer Vision: Motion & Tracking
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Tracking, profiling, and countermeasures against them
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Efficient visual trackers overfit to their training distributions and lack generalization abilities, resulting in them performing well on their respective in-distribution (ID) test sets and not as well on out-of-distribution (OOD) sequences, imposing limitations to their deployment in-the-wild under constrained resources. We introduce SiamABC, a highly efficient Siamese tracker that significantly improves tracking performance, even on OOD sequences. SiamABC takes advantage of new architectural designs in the way it bridges the dynamic variability of the target, and of new losses for training. Also, it directly addresses OOD tracking generalization by including a fast backward-free dynamic test-time adaptation method that continuously adapts the model according to the dynamic visual changes of the target. Our extensive experiments suggest that SiamABC shows remarkable performance gains in OOD sets while maintaining accurate performance on the ID benchmarks. SiamABC outperforms MixFormerV2-S by 7.6% on the OOD AVisT benchmark while being 3x faster (100 FPS) on a CPU. Our code and models are available at https://wvuvl.github.io/SiamABC/.

Problem

• Improves visual tracking generalization
• Addresses overfitting in tracking models
• Enhances out-of-distribution tracking efficiency

Innovation

• Siamese tracker architecture
• Dynamic test-time adaptation
• Improved OOD generalization