Similarity-Aware Machine Unlearning

2026-07-31

AI Summary

This work addresses a critical limitation in existing machine unlearning methods, which often neglect the impact of parameter modifications on retained data, thereby inflicting collateral damage on semantically similar samples that should be preserved. To mitigate this issue, the authors propose a retention-aware parameter localization strategy that identifies key parameters jointly influencing both the forget set and the retain set, enabling precise unlearning. They further introduce an evaluation set constructed via cosine similarity in the embedding space to quantitatively measure such collateral damage. Integrating parameter importance analysis with localized fine-tuning, the method demonstrates consistent improvements across eleven experiments on CIFAR-10 with ResNet-18, significantly reducing collateral damage while simultaneously enhancing standard unlearning metrics—thus achieving a favorable balance between effective forgetting and model fidelity.

Abstract

Machine unlearning removes the influence of user-specified training examples from a trained model, avoiding the need to retrain it from scratch. Localization-based methods improve unlearning efficiency by identifying a subset of influential model parameters. However, existing approaches select parameters based solely on forget-set importance, neglecting their role in retained dataset and often causing collateral damage to semantically similar retained examples. We address this limitation with a retain-aware localization method that considers parameter importance to both forgotten and retained data. We also introduce a retain-similar evaluation set, constructed using cosine similarity in the model embedding space, to directly measure collateral damage. Across eleven experimental settings on CIFAR-10 dataset and ResNet18 model, our method consistently reduces collateral damage while improving standard unlearning metrics, demonstrating the effectiveness of retain-aware localization for similarity-aware machine unlearning.

Problem

• machine unlearning
• collateral damage
• retain-aware
• similarity-aware
• parameter localization

Innovation

• machine unlearning
• retain-aware localization
• collateral damage
• similarity-aware
• parameter importance