PRUNE: A Patching Based Repair Framework for Certiffable Unlearning of Neural Networks

2025-05-10

AI Summary

To address the high retraining cost and verification difficulty in machine unlearning, this paper proposes a lightweight, verifiable neural network patching framework for forgetting. Methodologically, it pioneers a provably correct forgetting theory grounded in the “repair” paradigm; introduces a minimal-patch strategy coupled with a gradient-guided iterative selection of representative samples to ensure strict forgetting guarantees while enhancing scalability; and integrates neural network repair, SMT-based satisfiability solving, and efficient patch optimization—enabling both single-point and batch data deletion without retraining. Experimentally, on multi-class benchmarks, the approach achieves high-fidelity forgetting (accuracy drop <1.2%), reduces verification overhead by 3–5×, and significantly lowers memory consumption compared to retraining and fine-tuning baselines.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Search and Optimization: Learning to Search
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

It is often desirable to remove (a.k.a. unlearn) a speciffc part of the training data from a trained neural network model. A typical application scenario is to protect the data holder's right to be forgotten, which has been promoted by many recent regulation rules. Existing unlearning methods involve training alternative models with remaining data, which may be costly and challenging to verify from the data holder or a thirdparty auditor's perspective. In this work, we provide a new angle and propose a novel unlearning approach by imposing carefully crafted"patch"on the original neural network to achieve targeted"forgetting"of the requested data to delete. Speciffcally, inspired by the research line of neural network repair, we propose to strategically seek a lightweight minimum"patch"for unlearning a given data point with certiffable guarantee. Furthermore, to unlearn a considerable amount of data points (or an entire class), we propose to iteratively select a small subset of representative data points to unlearn, which achieves the effect of unlearning the whole set. Extensive experiments on multiple categorical datasets demonstrates our approach's effectiveness, achieving measurable unlearning while preserving the model's performance and being competitive in efffciency and memory consumption compared to various baseline methods.

Problem

• Remove specific training data from neural networks
• Achieve certifiable unlearning with lightweight patches
• Unlearn large data sets efficiently via representative subsets

Innovation

• Uses patching to unlearn specific data points
• Certifiably guarantees targeted forgetting with patches
• Iteratively unlearns representative subsets for efficiency