Technology Category
Machine Learning: Adversarial Learning & RobustnessComputer Vision: Adversarial Attacks & RobustnessSearch and Optimization: Learning to Search
Zero-Shot Machine Unlearning with Proxy Adversarial Data Generation
🤖 AI Summary
This paper addresses the over-forgetting problem in zero-shot machine unlearning, where access to retained data is unavailable. We propose a secure unlearning framework that requires no original retained data. Methodologically, we first generate adversarial proxy data to approximate the retained data distribution; second, we identify a sensitive subspace in parameter space associated with the to-be-forgotten samples and perform targeted optimization within this subspace; third, we introduce an influence-based pseudo-labeling strategy to mitigate performance degradation. Our approach integrates adversarial generation, subspace regularization, and influence-driven pseudo-labeling to achieve precise forgetting without compromising the utility of retained data. Extensive experiments across multiple benchmarks demonstrate that our method significantly outperforms existing zero-shot unlearning baselines. Theoretical analysis ensures rigor, while empirical results validate effectiveness and practicality.
Technology Category
Application Category
📝 Abstract
Machine unlearning aims to remove the influence of specific samples from a trained model. A key challenge in this process is over-unlearning, where the model's performance on the remaining data significantly drops due to the change in the model's parameters. Existing unlearning algorithms depend on the remaining data to prevent this issue. As such, these methods are inapplicable in a more practical scenario, where only the unlearning samples are available (i.e., zero-shot unlearning). This paper presents a novel framework, ZS-PAG, to fill this gap. Our approach offers three key innovations: (1) we approximate the inaccessible remaining data by generating adversarial samples; (2) leveraging the generated samples, we pinpoint a specific subspace to perform the unlearning process, therefore preventing over-unlearning in the challenging zero-shot scenario; and (3) we consider the influence of the unlearning process on the remaining samples and design an influence-based pseudo-labeling strategy. As a result, our method further improves the model's performance after unlearning. The proposed method holds a theoretical guarantee, and experiments on various benchmarks validate the effectiveness and superiority of our proposed method over several baselines.
Problem
Research questions and friction points this paper is trying to address.
Addresses over-unlearning in zero-shot scenarios
Generates adversarial samples to approximate remaining data
Improves model performance with influence-based pseudo-labeling
Innovation
Methods, ideas, or system contributions that make the work stand out.
Generates adversarial samples for data approximation
Identifies subspace to prevent over-unlearning
Uses influence-based pseudo-labeling for performance
🔎 Similar Papers
No similar papers found.
