A Robust Certified Machine Unlearning Method Under Distribution Shift

2026-01-11

arXiv.org

AI Summary

This work addresses the significant performance degradation of existing certified unlearning methods under non-i.i.d. deletion requests, which arises from distributional shifts. To tackle this challenge, we propose the first distribution-aware certified unlearning framework tailored for non-i.i.d. deletion scenarios. Our approach leverages iterative Newton updates constrained within a trust region to effectively approximate the fully retrained model, thereby achieving efficient (ε, δ)-certified unlearning. By deriving a tighter pre-run bound on the gradient residual, the method substantially mitigates the adverse effects induced by distributional shift. Extensive experiments demonstrate that our framework consistently outperforms current certified unlearning techniques across multiple evaluation metrics in settings with distributional shift.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Sequential Decision Making
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

The Newton method has been widely adopted to achieve certified unlearning. A critical assumption in existing approaches is that the data requested for unlearning are selected i.i.d.(independent and identically distributed). However,the problem of certified unlearning under non-i.i.d. deletions remains largely unexplored. In practice, unlearning requests are inherently biased, leading to non-i.i.d. deletions and causing distribution shifts between the original and retained datasets. In this paper, we show that certified unlearning with the Newton method becomes inefficient and ineffective under non-i.i.d. unlearning sets. We then propose a better certified unlearning approach by performing a distribution-aware certified unlearning framework based on iterative Newton updates constrained by a trust region. Our method provides a closer approximation to the retrained model and yields a tighter pre-run bound on the gradient residual, thereby ensuring efficient (epsilon, delta)-certified unlearning. To demonstrate its practical effectiveness under distribution shift, we also conduct extensive experiments across multiple evaluation metrics, providing a comprehensive assessment of our approach.

Problem

• certified machine unlearning
• distribution shift
• non-i.i.d. deletions
• Newton method
• data bias

Innovation

• certified machine unlearning
• distribution shift
• non-i.i.d. deletion
• trust region
• Newton method