Image-Text-Image Knowledge Transferring for Lifelong Person Re-Identification with Hybrid Clothing States

2024-05-26

arXiv.org

AI Summary

Existing lifelong person re-identification (LReID) methods overlook the practical challenge of frequent clothing changes. To address this, we introduce LReID-Hybrid—a novel lifelong re-identification task under hybrid clothing states—enabling dynamic co-evolution of same-clothing and cross-clothing domains during continual learning. We propose Teata, an image–text–image closed-loop knowledge transfer framework: (i) Structured Semantic Prompting (SSP) achieves fine-grained image–text alignment; (ii) Knowledge Adaptation and Projection (KAP) mitigates catastrophic forgetting. Teata integrates multimodal representation learning, slow-learner-driven parameter tuning, and cross-modal distillation. Extensive experiments demonstrate state-of-the-art performance on both the proposed LReID-Hybrid benchmark and standard LReID benchmarks, validating the effectiveness of hybrid clothing modeling and text-space knowledge relaying.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

With the continuous expansion of intelligent surveillance networks, lifelong person re-identification (LReID) has received widespread attention, pursuing the need of self-evolution across different domains. However, existing LReID studies accumulate knowledge with the assumption that people would not change their clothes. In this paper, we propose a more practical task, namely lifelong person re-identification with hybrid clothing states (LReID-Hybrid), which takes a series of cloth-changing and cloth-consistent domains into account during lifelong learning. To tackle the challenges of knowledge granularity mismatch and knowledge presentation mismatch that occurred in LReID-Hybrid, we take advantage of the consistency and generalization of the text space, and propose a novel framework, dubbed $Teata$, to effectively align, transfer and accumulate knowledge in an"image-text-image"closed loop. Concretely, to achieve effective knowledge transfer, we design a Structured Semantic Prompt (SSP) learning to decompose the text prompt into several structured pairs to distill knowledge from the image space with a unified granularity of text description. Then, we introduce a Knowledge Adaptation and Projection strategy (KAP), which tunes text knowledge via a slow-paced learner to adapt to different tasks without catastrophic forgetting. Extensive experiments demonstrate the superiority of our proposed $Teata$ for LReID-Hybrid as well as on conventional LReID benchmarks over advanced methods.

Problem

• Lifelong person re-identification with clothing changes
• Knowledge transfer between image and text spaces
• Preventing catastrophic forgetting across domains

Innovation

• Image-text-image closed loop transfer
• Structured Semantic Prompt learning
• Knowledge Adaptation and Projection strategy