What really matters for person re-identification? A Mixture-of-Experts Framework for Semantic Attribute Importance

2025-12-09

AI Summary

Pedestrian re-identification (ReID) models lack interpretable attribution of their reliance on high-level semantic attributes (e.g., clothing color, accessories). To address this, we propose MoSAIC-ReID—a novel Mixture-of-Semantic-Attribute-Interpretable-Experts framework for controllable attribution analysis. It integrates lightweight LoRA-based expert networks with an Oracle routing mechanism and couples generalized linear modeling with statistical hypothesis testing to enable large-scale, quantitative assessment of semantic attribute importance. Evaluated on Market-1501 and DukeMTMC, our method reveals that frequent attributes—particularly clothing color—dominate model decisions, whereas rare attributes exhibit limited contribution. This work establishes the first principled, systematic intrinsic–extrinsic feature attribution methodology for ReID, enabling fine-grained, semantically grounded interpretability. By explicitly quantifying how visual attributes influence identity matching, MoSAIC-ReID significantly enhances model transparency, trustworthiness, and diagnostic capability—advancing the field toward accountable and human-understandable ReID systems.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Mixture of Experts (MoE)
• Knowledge Representation and Reasoning: Common-Sense Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

State-of-the-art person re-identification methods achieve impressive accuracy but remain largely opaque, leaving open the question: which high-level semantic attributes do these models actually rely on? We propose MoSAIC-ReID, a Mixture-of-Experts framework that systematically quantifies the importance of pedestrian attributes for re-identification. Our approach uses LoRA-based experts, each linked to a single attribute, and an oracle router that enables controlled attribution analysis. While MoSAIC-ReID achieves competitive performance on Market-1501 and DukeMTMC under the assumption that attribute annotations are available at test time, its primary value lies in providing a large-scale, quantitative study of attribute importance across intrinsic and extrinsic cues. Using generalized linear models, statistical tests, and feature-importance analyses, we reveal which attributes, such as clothing colors and intrinsic characteristics, contribute most strongly, while infrequent cues (e.g. accessories) have limited effect. This work offers a principled framework for interpretable ReID and highlights the requirements for integrating explicit semantic knowledge in practice. Code is available at https://github.com/psaltaath/MoSAIC-ReID

Problem

• Quantifies semantic attribute importance for person re-identification models
• Analyzes which pedestrian attributes contribute most to re-identification accuracy
• Provides interpretable framework to understand model reliance on visual cues

Innovation

• Mixture-of-Experts framework quantifies attribute importance
• LoRA-based experts linked to attributes enable controlled analysis
• Statistical analyses reveal key attributes like clothing colors