Cross-Modality Perturbation Synergy Attack for Person Re-identification

2024-01-18

Neural Information Processing Systems

AI Summary

Existing security research on person re-identification (ReID) focuses predominantly on single-modality (RGB) settings, leaving cross-modal (e.g., RGB-infrared) ReID systems’ vulnerabilities systematically unexplored. Method: This work presents the first systematic security study of cross-modal ReID models and proposes a modality-aware universal adversarial attack. By modeling cross-modal feature disentanglement, the method jointly exploits gradient signals from both infrared and visible-light modalities to co-optimize a universal perturbation—thereby amplifying inter-modal feature discrepancy and degrading discriminative capability. Contribution/Results: The approach explicitly accounts for modality-specific characteristics and distributional shifts between modalities. Evaluated on three major benchmarks—RegDB, SYSU-MM01, and LLCM—it reduces Rank-1 accuracy by over 40% on average, exposing critical robustness bottlenecks in current cross-modal ReID architectures. This work establishes a novel paradigm for security assessment of multimodal vision systems.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Large-scale security measurements

Abstract

In recent years, there has been significant research focusing on addressing security concerns in single-modal person re-identification (ReID) systems that are based on RGB images. However, the safety of cross-modality scenarios, which are more commonly encountered in practical applications involving images captured by infrared cameras, has not received adequate attention. The main challenge in cross-modality ReID lies in effectively dealing with visual differences between different modalities. For instance, infrared images are typically grayscale, unlike visible images that contain color information. Existing attack methods have primarily focused on the characteristics of the visible image modality, overlooking the features of other modalities and the variations in data distribution among different modalities. This oversight can potentially undermine the effectiveness of these methods in image retrieval across diverse modalities. This study represents the first exploration into the security of cross-modality ReID models and proposes a universal perturbation attack specifically designed for cross-modality ReID. This attack optimizes perturbations by leveraging gradients from diverse modality data, thereby disrupting the discriminator and reinforcing the differences between modalities. We conducted experiments on three widely used cross-modality datasets, namely RegDB, SYSU, and LLCM. The results not only demonstrate the effectiveness of our method but also provide insights for future improvements in the robustness of cross-modality ReID systems. The code will be available at https://github.com/finger-monkey/cmps__attack.

Problem

• Addressing security gaps in cross-modality person ReID systems
• Overcoming visual differences between RGB and infrared modalities
• Developing universal perturbation attacks for multi-modal ReID robustness

Innovation

• Universal perturbation attack for cross-modality ReID
• Optimizes perturbations using diverse modality gradients
• Disrupts discriminator and reinforces modality differences