🤖 AI Summary
Conventional machine unlearning methods for continual learning (CL) rely on access to historical training data, leading to catastrophic forgetting of previously learned tasks and task recurrence. Method: This paper proposes UnCLe, a data-free unlearning framework that integrates unlearning directly into the CL paradigm. UnCLe employs a hypernetwork conditioned on task embeddings to generate task-specific parameters and introduces a noise-alignment strategy to enable selective, data-agnostic unlearning—without requiring original training data. Contribution/Results: By decoupling unlearning from data dependency, UnCLe preserves forward transfer capability while effectively mitigating knowledge interference and backward task reactivation. Extensive evaluations on multiple visual benchmarks demonstrate that UnCLe supports joint optimization of incremental learning and unlearning across multiple rounds, significantly improving model stability and post-unlearning accuracy retention compared to state-of-the-art baselines.
📝 Abstract
Growing concerns surrounding AI safety and data privacy have driven the development of Machine Unlearning as a potential solution. However, current machine unlearning algorithms are designed to complement the offline training paradigm. The emergence of the Continual Learning (CL) paradigm promises incremental model updates, enabling models to learn new tasks sequentially. Naturally, some of those tasks may need to be unlearned to address safety or privacy concerns that might arise. We find that applying conventional unlearning algorithms in continual learning environments creates two critical problems: performance degradation on retained tasks and task relapse, where previously unlearned tasks resurface during subsequent learning. Furthermore, most unlearning algorithms require data to operate, which conflicts with CL's philosophy of discarding past data. A clear need arises for unlearning algorithms that are data-free and mindful of future learning. To that end, we propose UnCLe, an Unlearning framework for Continual Learning. UnCLe employs a hypernetwork that learns to generate task-specific network parameters, using task embeddings. Tasks are unlearned by aligning the corresponding generated network parameters with noise, without requiring any data. Empirical evaluations on several vision data sets demonstrate UnCLe's ability to sequentially perform multiple learning and unlearning operations with minimal disruption to previously acquired knowledge.