Multi-Modal Object Re-Identification with Dual Semantic Guidance and Global-Local Mutual Modulation

2026-07-31

AI Summary

This work addresses the challenges of background interference and cross-modal misalignment in multimodal person re-identification, which stem from insufficient semantic prior alignment and inadequate collaboration between global and local features. To tackle these issues, we propose a dual semantic-guided and global-local mutual modulation framework that integrates Text–Visual Semantic Injection (TSI), Masked Global–Local Modulator (MGLM), and Hierarchical Mixture-of-Experts Fusion (HMF). This approach enables fine-grained, semantics-driven feature alignment and adaptive aggregation by jointly leveraging clean textual semantics, soft-masked local perception, and a hierarchical expert mechanism—marking the first such integration in this domain. The proposed method achieves state-of-the-art performance on three multimodal ReID benchmarks, demonstrating significantly enhanced cross-modal robustness and discriminability. The code is publicly available.

Abstract

Multi-modal object Re-Identification (ReID) aims to retrieve target instances by leveraging complementary information across modalities. However, existing methods suffer from two challenges. First, they often fail to exploit well-aligned and reliable semantic priors, making them vulnerable to background clutter and cross-modal misalignment. On the other hand, they typically rely on holistic feature modeling, overlooking the synergy between global and local representations. To overcome these limitations, we propose a robust multi-modal ReID framework with dual semantic guidance and global-local mutual modulation, which mainly consists of three key components, namely the Text-Semantic Injector (TSI), the Masked Global-Local Modulator (MGLM), and the Hierarchical MoE Fusion (HMF). The TSI enhances semantic awareness by integrating clean and coherent textual features into visual tokens. The MGLM enables part-aware cross-modal interaction through joint guidance from soft masks and global context, improving fine-grained feature alignment. Finally, the HMF adaptively aggregates multi-spectral features under local semantic supervision, yielding discriminative and robust representations. Extensive experiments on three multi-modal ReID benchmarks demonstrate the effectiveness of the proposed method. The code will be made publicly available at https://github.com/zw-absin/DSGM upon acceptance.

Problem

• Multi-modal Re-Identification
• semantic priors
• cross-modal misalignment
• global-local representation
• background clutter

Innovation

• multi-modal ReID
• semantic guidance
• global-local modulation
• text-visual alignment
• feature fusion