Exploring Robustness of Visual State Space model against Backdoor Attacks

2024-08-21

arXiv.org

AI Summary

Visual State Space Models (VSSMs) exhibit significantly lower robustness against backdoor attacks than gated CNNs—though outperforming ViTs—primarily because the SSM’s explicit modeling of local patch-wise context amplifies sensitivity to block-level perturbations. Method: We propose, for the first time, an *intra-block repeated trigger strategy*, specifically tailored to VSSM architectural characteristics, which departs from conventional single-point triggering paradigms. We conduct comprehensive cross-dataset backdoor attacks and defenses on CIFAR-10, CIFAR-100, and ImageNet-1K, incorporating diverse trigger injection methods and fine-grained block-level sensitivity analysis. Contribution/Results: Our strategy substantially degrades the efficacy of mainstream backdoor attacks across all benchmarks. Empirical results confirm that mitigating block-level perturbation vulnerability is pivotal to enhancing VSSM robustness. This work establishes a novel paradigm for security evaluation and defense design targeting sequence-based vision architectures.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Visual State Space Model (VSS) has demonstrated remarkable performance in various computer vision tasks. However, in the process of development, backdoor attacks have brought severe challenges to security. Such attacks cause an infected model to predict target labels when a specific trigger is activated, while the model behaves normally on benign samples. In this paper, we conduct systematic experiments to comprehend on robustness of VSS through the lens of backdoor attacks, specifically how the state space model (SSM) mechanism affects robustness. We first investigate the vulnerability of VSS to different backdoor triggers and reveal that the SSM mechanism, which captures contextual information within patches, makes the VSS model more susceptible to backdoor triggers compared to models without SSM. Furthermore, we analyze the sensitivity of the VSS model to patch processing techniques and discover that these triggers are effectively disrupted. Based on these observations, we consider an effective backdoor for the VSS model that recurs in each patch to resist patch perturbations. Extensive experiments across three datasets and various backdoor attacks reveal that the VSS model performs comparably to Transformers (ViTs) but is less robust than the Gated CNNs, which comprise only stacked Gated CNN blocks without SSM.

Problem

• Investigating backdoor attack vulnerabilities in Visual State Space Models
• Developing BadVim framework using low-rank perturbations on state transitions
• Demonstrating high attack success rates with minimal poisoned training data

Innovation

• Low-rank perturbations on state-wise transitions
• Poisoning only 0.3% of training data
• Bypassing state-of-the-art backdoor defenses