Towards Certification of Uncertainty Calibration under Adversarial Attacks

2024-05-22

arXiv.org

AI Summary

Neural networks suffer not only accuracy degradation but also severe deterioration in uncertainty calibration—measured by metrics such as the Brier score and Expected Calibration Error (ECE)—under adversarial perturbations, undermining trustworthy decision-making in safety-critical applications. This work formally introduces the problem of *certified calibration*, deriving worst-case theoretical robustness bounds for both the Brier score and ECE: an analytical upper bound for the Brier score and a mixed-integer programming (MIP)-based approximate upper bound for ECE. We further design a novel calibration-directed adversarial attack and propose an adversarial calibration training paradigm. Experiments demonstrate that calibration degradation is quantitatively characterizable, and the derived certified bounds tightly reflect empirical deterioration. The proposed training method substantially improves calibration robustness across diverse adversarial attacks, with consistent efficacy and generalizability validated on multiple benchmark datasets.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Adversarial Attacks & Robustness
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Since neural classifiers are known to be sensitive to adversarial perturbations that alter their accuracy, extit{certification methods} have been developed to provide provable guarantees on the insensitivity of their predictions to such perturbations. Furthermore, in safety-critical applications, the frequentist interpretation of the confidence of a classifier (also known as model calibration) can be of utmost importance. This property can be measured via the Brier score or the expected calibration error. We show that attacks can significantly harm calibration, and thus propose certified calibration as worst-case bounds on calibration under adversarial perturbations. Specifically, we produce analytic bounds for the Brier score and approximate bounds via the solution of a mixed-integer program on the expected calibration error. Finally, we propose novel calibration attacks and demonstrate how they can improve model calibration through extit{adversarial calibration training}.

Problem

• Certify neural classifier uncertainty calibration
• Provide bounds under adversarial perturbations
• Propose adversarial calibration training methods

Innovation

• Certified calibration under adversarial attacks
• Analytic bounds for Brier score
• Adversarial calibration training improves models