Causal Unlearning in Collaborative Optimization: Exact and Approximate Influence Reversal under Adversarial Contributions

2026-05-19

AI Summary

This work addresses the challenge of efficiently fulfilling data deletion requests in federated learning without incurring prohibitive retraining costs. The authors propose a Krylov subspace-based conjugate gradient approximation method to accurately estimate the influence of individual clients on the global model. Coupled with a causal weighting mechanism, the approach enables “surgical” unlearning by updating only those parameters affected by the deleted data. The method simultaneously preserves adversarial robustness and achieves strong privacy guarantees: on benchmarks such as CIFAR-10, it accelerates deletion by 47.75× compared to full retraining, incurs no more than a 0.60% drop in test accuracy, and reduces membership inference attack success rates to 0.499—approaching the privacy level of complete retraining.

Abstract

Federated learning systems must support data deletion requests to comply with privacy regulations, yet retraining from scratch after each deletion is computationally prohibitive. We present HF-KCU, a method that removes a client's contribution by approximating the influence function through conjugate gradient iterations in Krylov subspaces, reducing complexity from O(d^3) to O(kd) where k<<d.A causal weighting mechanism ensures that only clients holding the deleted data receive parameter updates, preventing spurious changes to unaffected clients. Our method is designed to handle bounded adversarial perturbations to the Hessian and gradient, providing graceful degradation under realistic threat models. We validate HF-KCU across convolutional (ResNet-18, SimpleCNN) and transformer (ViT-Lite) architectures on CIFAR-10, MNIST, and Fashion-MNIST. On CIFAR-10 under Dirichlet (alpha=0.5) partitioning, HF-KCU achieves 47.75 times speedup over retraining while maintaining test accuracy within 0.60% of the rational baseline(71.16 vs 71.76 %). Membership inference attacks on the forget set yield success rates of 0.499 matching the retrained model and confirming effective privacy restoration. We provide convergence guarantees showing that the Krylov approximation error decreases as O((k ^1/2-1)/(k^1/2+1)) where k is the Hessian condition number. The causal weighting mechanism ensures surgical updates, where only clients holding deleted data are modified, preserving model quality for unaffected participants and avoiding the instability of gradient-based approaches in asynchronous federated settings. This design provides interpretability as each update is directly traceable to the influence of the deleted data. The method's efficiency and precision make it suitable for production federated systems where deletion requests arrive asynchronously and computational budgets are constrained.

Problem

• federated learning
• data deletion
• causal unlearning
• adversarial robustness
• privacy compliance

Innovation

• Causal Unlearning
• Federated Learning
• Influence Function Approximation
• Krylov Subspace
• Adversarial Robustness