Patch Synthesis for Property Repair of Deep Neural Networks

2024-04-02

AI Summary

Deep neural networks often lack local robustness in safety-critical applications, and existing repair methods suffer from limited generality and absence of formal guarantees. Method: This paper proposes a modular, property-level provably correct repair framework. It formally models robust neighborhoods, synthesizes lightweight patch modules, and integrates heuristic patch assignment with locally sensitive parameter recalibration—ensuring robustness across all inputs within the neighborhood without degrading original model performance. Contribution/Results: The work introduces the first property-level repair paradigm, achieving 100% coverage with formal verification for high-dimensional data. It provides rigorous theoretical guarantees while maintaining computational efficiency and deployability. Extensive evaluation on multiple benchmark datasets demonstrates significant improvements over state-of-the-art methods, with superior scalability and cross-sample generalization capability.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Deep neural networks (DNNs) are prone to various dependability issues, such as adversarial attacks, which hinder their adoption in safety-critical domains. Recently, NN repair techniques have been proposed to address these issues while preserving original performance by locating and modifying guilty neurons and their parameters. However, existing repair approaches are often limited to specific data sets and do not provide theoretical guarantees for the effectiveness of the repairs. To address these limitations, we introduce PatchPro, a novel patch-based approach for property-level repair of DNNs, focusing on local robustness. The key idea behind PatchPro is to construct patch modules that, when integrated with the original network, provide specialized repairs for all samples within the robustness neighborhood while maintaining the network's original performance. Our method incorporates formal verification and a heuristic mechanism for allocating patch modules, enabling it to defend against adversarial attacks and generalize to other inputs. PatchPro demonstrates superior efficiency, scalability, and repair success rates compared to existing DNN repair methods, i.e., realizing provable property-level repair for 100% cases across multiple high-dimensional datasets.

Problem

• Deep Neural Networks
• Adversarial Attacks
• Vulnerability Mitigation

Innovation

• PatchPro
• Adversarial Attacks
• Neural Network Repair