Robustness of deep learning classification to adversarial input on GPUs: asynchronous parallel accumulation is a source of vulnerability

2025-03-21

AI Summary

This work reveals that floating-point non-associativity (FPNA) induced by GPU asynchronous parallel accumulation can cause misclassification in deep learning classifiers—even without input perturbations—particularly for samples near decision boundaries, leading to systematic overestimation of standard adversarial robustness (up to 4.6%). To address this, we propose three key contributions: (1) the first identification of GPU-level asynchronous parallelism as a novel hardware-layer adversarial vulnerability; (2) a learnable permutation (LP) gradient method that efficiently estimates the worst-case FPNA impact across thousands of GPU execution states with minimal overhead; and (3) a Bayesian optimization–based black-box attack framework empirically validating how background load, virtualization, and power management bias reduction outcomes. Cross-architecture experiments demonstrate that reduction-order sensitivity is highly dependent on runtime hardware conditions.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The ability of machine learning (ML) classification models to resist small, targeted input perturbations - known as adversarial attacks - is a key measure of their safety and reliability. We show that floating-point non associativity (FPNA) coupled with asynchronous parallel programming on GPUs is sufficient to result in misclassification, without any perturbation to the input. Additionally, we show this misclassification is particularly significant for inputs close to the decision boundary and that standard adversarial robustness results may be overestimated up to 4.6% when not considering machine-level details. We first study a linear classifier, before focusing on standard Graph Neural Network (GNN) architectures and datasets. We present a novel black-box attack using Bayesian optimization to determine external workloads that bias the output of reductions on GPUs and reliably lead to misclassification. Motivated by these results, we present a new learnable permutation (LP) gradient-based approach, to learn floating point operation orderings that lead to misclassifications, making the assumption that any reduction or permutation ordering is possible. This LP approach provides a worst-case estimate in a computationally efficient manner, avoiding the need to run identical experiments tens of thousands of times over a potentially large set of possible GPU states or architectures. Finally, we investigate parallel reduction ordering across different GPU architectures for a reduction under three conditions: (1) executing external background workloads, (2) utilizing multi-GPU virtualization, and (3) applying power capping. Our results demonstrate that parallel reduction ordering varies significantly across architectures under the first two conditions. The results and methods developed here can help to include machine-level considerations into adversarial robustness assessments.

Problem

• Investigates GPU-induced misclassification in ML models without input perturbations
• Examines adversarial robustness overestimation due to floating-point non-associativity
• Proposes learnable permutation method to assess worst-case misclassification scenarios

Innovation

• Bayesian optimization for black-box GPU attacks
• Learnable permutation gradient-based approach
• Analyzing parallel reduction across GPU architectures