On the Robustness of Adversarial Training Against Uncertainty Attacks

2024-10-29

arXiv.org

AI Summary

In safety-critical applications, adversarial attacks can maliciously manipulate model uncertainty estimates—over- or under-estimating confidence—thereby undermining decision reliability and system usability. This work is the first to theoretically and empirically demonstrate that standard adversarial training methods (e.g., TRADES, PGD) inherently improve the robustness of uncertainty estimation, obviating the need for dedicated uncertainty-specific defenses. We conduct a systematic evaluation across multiple adversarially robust models on CIFAR-10 and ImageNet using the RobustBench benchmark, showing substantial gains in resilience against uncertainty-targeted attacks: uncertainty calibration error decreases by up to 42% compared to standard models. We further provide theoretical analysis proving that this robustness arises from an implicit regularization effect induced by adversarial perturbations on the confidence margin—effectively tightening the bounds of predictive confidence. Our findings bridge adversarial robustness and reliable uncertainty quantification, offering a principled, unified approach to trustworthy AI in high-stakes settings.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Reasoning under Uncertainty: Uncertainty Representations

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In learning problems, the noise inherent to the task at hand hinders the possibility to infer without a certain degree of uncertainty. Quantifying this uncertainty, regardless of its wide use, assumes high relevance for security-sensitive applications. Within these scenarios, it becomes fundamental to guarantee good (i.e., trustworthy) uncertainty measures, which downstream modules can securely employ to drive the final decision-making process. However, an attacker may be interested in forcing the system to produce either (i) highly uncertain outputs jeopardizing the system's availability or (ii) low uncertainty estimates, making the system accept uncertain samples that would instead require a careful inspection (e.g., human intervention). Therefore, it becomes fundamental to understand how to obtain robust uncertainty estimates against these kinds of attacks. In this work, we reveal both empirically and theoretically that defending against adversarial examples, i.e., carefully perturbed samples that cause misclassification, additionally guarantees a more secure, trustworthy uncertainty estimate under common attack scenarios without the need for an ad-hoc defense strategy. To support our claims, we evaluate multiple adversarial-robust models from the publicly available benchmark RobustBench on the CIFAR-10 and ImageNet datasets.

Problem

• Ensuring robust uncertainty estimates against adversarial attacks
• Defending adversarial examples for trustworthy uncertainty measures
• Evaluating adversarial-robust models on security-sensitive applications

Innovation

• Adversarial training ensures robust uncertainty estimates
• Defends against adversarial examples without ad-hoc strategies
• Evaluated on CIFAR-10 and ImageNet datasets