Reliable Multi-Modal Object Re-Identification via Modality-Aware Graph Reasoning

2025-04-21

AI Summary

To address uneven local feature quality, insufficient cross-modal complementary information exploitation, and poor robustness to missing modalities in multimodal person re-identification (ReID), this paper proposes a modality-aware graph learning framework. It constructs a modality-adaptive graph structure and introduces a selective node exchange mechanism to enhance high-quality local features while suppressing low-confidence responses. A local-aware graph reasoning module and a cross-modal information propagation module are designed to explicitly model inter-modal synergies. Moreover, this work is the first to leverage graph-structured representations for implicit reconstruction of missing modalities in ReID, thereby mitigating modality-specific representation bias. The proposed method achieves state-of-the-art performance on four major benchmarks—RGBNT201, Market1501-MM, RGBNT100, and MSVR310—demonstrating significant improvements in robustness and generalization under complex, real-world scenarios.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Multi-modal data provides abundant and diverse object information, crucial for effective modal interactions in Re-Identification (ReID) tasks. However, existing approaches often overlook the quality variations in local features and fail to fully leverage the complementary information across modalities, particularly in the case of low-quality features. In this paper, we propose to address this issue by leveraging a novel graph reasoning model, termed the Modality-aware Graph Reasoning Network (MGRNet). Specifically, we first construct modality-aware graphs to enhance the extraction of fine-grained local details by effectively capturing and modeling the relationships between patches. Subsequently, the selective graph nodes swap operation is employed to alleviate the adverse effects of low-quality local features by considering both local and global information, enhancing the representation of discriminative information. Finally, the swapped modality-aware graphs are fed into the local-aware graph reasoning module, which propagates multi-modal information to yield a reliable feature representation. Another advantage of the proposed graph reasoning approach is its ability to reconstruct missing modal information by exploiting inherent structural relationships, thereby minimizing disparities between different modalities. Experimental results on four benchmarks (RGBNT201, Market1501-MM, RGBNT100, MSVR310) indicate that the proposed method achieves state-of-the-art performance in multi-modal object ReID. The code for our method will be available upon acceptance.

Problem

• Addresses quality variations in local features for multi-modal ReID
• Enhances complementary information use across different modalities
• Reconstructs missing modal information via structural relationships

Innovation

• Modality-aware graphs enhance fine-grained local details
• Selective graph nodes swap improves low-quality features
• Local-aware reasoning propagates multi-modal information effectively