Impactful Bit-Flip Search on Full-precision Models

2024-11-12

arXiv.org

AI Summary

Neural networks are vulnerable to bit-flip attacks (BFAs), where flipping a small number of critical parameter bits can severely degrade model performance; however, efficiently identifying such impactful bits in full-precision models remains challenging. This paper proposes Impactful Bit-Flip Search (IBS), the first BFA search framework tailored for full-precision models. IBS integrates sensitivity-driven bit importance scoring, layer-wise progressive search, and a Weight-Stealth weight-tuning mechanism that preserves the original floating-point value distribution—thereby evading range-based integrity checks. Experiments demonstrate that flipping merely 0.001% of parameter bits reduces top-1 accuracy by over 50% across multiple mainstream full-precision models, significantly outperforming existing BFA methods. IBS establishes a new benchmark for robustness analysis and adversarial defense in full-precision deep learning systems.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Neural networks have shown remarkable performance in various tasks, yet they remain susceptible to subtle changes in their input or model parameters. One particularly impactful vulnerability arises through the Bit-Flip Attack (BFA), where flipping a small number of critical bits in a model's parameters can severely degrade its performance. A common technique for inducing bit flips in DRAM is the Row-Hammer attack, which exploits frequent uncached memory accesses to alter data. Identifying susceptible bits can be achieved through exhaustive search or progressive layer-by-layer analysis, especially in quantized networks. In this work, we introduce Impactful Bit-Flip Search (IBS), a novel method for efficiently pinpointing and flipping critical bits in full-precision networks. Additionally, we propose a Weight-Stealth technique that strategically modifies the model's parameters in a way that maintains the float values within the original distribution, thereby bypassing simple range checks often used in tamper detection.

Problem

• Identify critical bits in neural networks
• Mitigate Bit-Flip Attack vulnerabilities
• Maintain model integrity with Weight-Stealth technique

Innovation

• Bit-Flip Search method
• Weight-Stealth technique
• full-precision networks analysis