Continual Data Unlearning in Diffusion Models via Transition-based Regularization

2026-09-26

AI Summary

This study addresses the challenges of utility degradation caused by new requests and early forgetting failure in continuous data unlearning for diffusion models. We propose a continual unlearning framework based on transition regularization. Specifically, this method pioneers the use of parameters from previously completed deletions as directional references and constructs a fixed-capacity memory bank by selecting representative samples via local sensitivity analysis, thereby decoupling storage overhead from request volume. Furthermore, a unidirectional penalty is imposed on parameter updates to preserve forgetting persistence. Experimental results demonstrate that, in cumulative deletion scenarios, our approach requires minimal memory while significantly improving the trade-off between forgetting durability and generative utility, outperforming existing baselines.

Abstract

Data unlearning in diffusion models aims to remove the influence of specific training examples without suppressing the broader concepts they represent. However, when deletion requests arrive sequentially, updates for new requests can degrade generative utility and undermine earlier deletions. We propose a continual data unlearning framework that uses completed deletion transitions as directional references to regularize future updates. For each request, we record changes in denoiser responses on the same fixed noisy inputs before and after unlearning. Rather than matching full post-deletion responses, we apply a one-sided penalty that discourages reversal along the recorded directions relative to the post-deletion references, while leaving orthogonal response changes and progress beyond these references unpenalized. To keep storage independent of the number of requests, we maintain a fixed-capacity bank of representative transition records. Records are selected based on the local sensitivity of progress along their recorded directions to parameter updates, allowing them to be retained even when their penalties are inactive. Empirical evaluations show that the proposed framework achieves a better balance between deletion persistence and generative utility than existing unlearning baselines as requests accumulate, using only a small transition memory.

Problem

• Continual Data Unlearning
• Diffusion Models
• Deletion Persistence
• Generative Utility
• Sequential Deletion Requests

Innovation

• Continual Data Unlearning
• Diffusion Models
• Transition-based Regularization
• One-sided Penalty
• Fixed-capacity Transition Bank