Modality Unified Attack for Omni-Modality Person Re-Identification

2025-01-22

AI Summary

Existing adversarial attacks on person re-identification (re-id) are limited to single- or cross-modal settings and fail to address robustness threats against black-box, omni-modality re-id systems—encompassing unimodal, cross-modal, and multimodal architectures. Method: We propose the first modality-agnostic universal adversarial attack framework, requiring no prior knowledge of the target model. It unifies cross-modal feature simulation perturbations with multimodal collaborative metric disruption, jointly optimized via a multimodal surrogate model, modality-specific generators, and a metric-disruption loss. Contribution/Results: Evaluated on four representative omni-modality re-id models, our method achieves average mAP degradation rates of 55.9%, 24.4%, 49.0%, and 62.7%, significantly outperforming state-of-the-art single- and cross-modal attacks. To the best of our knowledge, this is the first work enabling universal, efficient, and dependency-free adversarial attacks against omni-modality re-id systems.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Deep learning based person re-identification (re-id) models have been widely employed in surveillance systems. Recent studies have demonstrated that black-box single-modality and cross-modality re-id models are vulnerable to adversarial examples (AEs), leaving the robustness of multi-modality re-id models unexplored. Due to the lack of knowledge about the specific type of model deployed in the target black-box surveillance system, we aim to generate modality unified AEs for omni-modality (single-, cross- and multi-modality) re-id models. Specifically, we propose a novel Modality Unified Attack method to train modality-specific adversarial generators to generate AEs that effectively attack different omni-modality models. A multi-modality model is adopted as the surrogate model, wherein the features of each modality are perturbed by metric disruption loss before fusion. To collapse the common features of omni-modality models, Cross Modality Simulated Disruption approach is introduced to mimic the cross-modality feature embeddings by intentionally feeding images to non-corresponding modality-specific subnetworks of the surrogate model. Moreover, Multi Modality Collaborative Disruption strategy is devised to facilitate the attacker to comprehensively corrupt the informative content of person images by leveraging a multi modality feature collaborative metric disruption loss. Extensive experiments show that our MUA method can effectively attack the omni-modality re-id models, achieving 55.9%, 24.4%, 49.0% and 62.7% mean mAP Drop Rate, respectively.

Problem

• Adversarial Samples
• Multi-modal Recognition
• Deep Learning Face Recognition

Innovation

• Modal-Unified Attack
• Adversarial Images
• Black-box Conditions