Multi-Modal Object Re-Identification with Prompt-S6 and Semantic-Aware Knowledge Guidance

2026-07-26

AI Summary

This work addresses the challenges in multimodal person re-identification, where existing methods struggle to suppress background interference, achieve effective trilateral modality alignment, and rely on computationally expensive pairwise feature fusion. To overcome these limitations, we propose PRISM, a novel framework that introduces Prompt-S6 for the first time to enable efficient cross-modal interaction with linear complexity. Leveraging semantic priors from foundation segmentation models, PRISM incorporates a Semantic-Driven Token Pruning (SDTP) mechanism to enhance robustness. Furthermore, a Progressive Fusion Network (PFN) is designed to hierarchically align features across the three modalities. Extensive experiments demonstrate that PRISM significantly outperforms state-of-the-art methods on four multimodal ReID benchmarks, achieving leading performance in both accuracy and computational efficiency.

Abstract

Multi-modal object Re-Identification (ReID) aims to retrieve specific objects by integrating complementary information from multiple modalities. However, existing multi-modal ReID methods do not effectively address background interference suppression or achieve tri-modal alignment, instead focusing on pairwise feature fusion. Moreover, many current aggregation approaches suffer from high computational complexity. To address these limitations, we propose PRISM, a novel multi-modal ReID framework built upon Prompt-S6 (PS6) and semantic-aware knowledge guidance. PS6 maintains the linear complexity and strong sequence modeling capability of Mamba while enabling efficient cross-modal interaction. Leveraging these advantages, we design two key components: Semantic-Driven Token Pruning (SDTP) and Progressive Fusion Network (PFN). Parsing semantic priors from the segmentation foundation models, the SDTP then leverages these priors and applies dynamic token pruning to suppress background noise and refine feature representations. The PFN progressively aggregates multi-modal features to achieve tri-modal alignment and fully exploit modality complementarity. With the proposed modules, PRISM generates more robust multi-modal representations under complex scenarios. Extensive experiments on four multi-modal object ReID benchmarks demonstrate the effectiveness and efficiency of our approach. The source code is available at https://github.com/zw-absin/PRISM.

Problem

• multi-modal Re-Identification
• background interference suppression
• tri-modal alignment
• computational complexity

Innovation

• Prompt-S6
• Semantic-Aware Knowledge Guidance
• Token Pruning
• Tri-Modal Alignment
• Multi-Modal Re-Identification