Data Unlearning Beyond Uniform Forgetting via Diffusion Time and Frequency Selection

2025-10-19

AI Summary

Data forgetting in diffusion models remains challenged by degraded generation quality and incomplete forgetting. This paper identifies, for the first time, pronounced spatiotemporal non-uniformity of forgetting effects—both across denoising timesteps and in the frequency domain—and proposes a timestep-frequency selective forgetting strategy to avoid the quality degradation inherent in uniform full-timestep forgetting. To enable precise evaluation, we extend the SSCD metric to jointly quantify forgetting completeness and generation fidelity. Our method integrates gradient-based optimization with preference learning objectives and is validated on both image-level and text-to-image generation tasks. Experiments demonstrate substantial improvements in post-forgetting sample aesthetics and denoising capability, achieving more complete and controllable data removal while preserving high-fidelity generation.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Data unlearning aims to remove the influence of specific training samples from a trained model without requiring full retraining. Unlike concept unlearning, data unlearning in diffusion models remains underexplored and often suffers from quality degradation or incomplete forgetting. To address this, we first observe that most existing methods attempt to unlearn the samples at all diffusion time steps equally, leading to poor-quality generation. We argue that forgetting occurs disproportionately across time and frequency, depending on the model and scenarios. By selectively focusing on specific time-frequency ranges during training, we achieve samples with higher aesthetic quality and lower noise. We validate this improvement by applying our time-frequency selective approach to diverse settings, including gradient-based and preference optimization objectives, as well as both image-level and text-to-image tasks. Finally, to evaluate both deletion and quality of unlearned data samples, we propose a simple normalized version of SSCD. Together, our analysis and methods establish a clearer understanding of the unique challenges in data unlearning for diffusion models, providing practical strategies to improve both evaluation and unlearning performance.

Problem

• Addressing quality degradation in diffusion model data unlearning
• Selectively targeting time-frequency ranges for effective forgetting
• Improving evaluation metrics for unlearning performance assessment

Innovation

• Selective time-frequency training for unlearning
• Applying method to gradient and preference optimization
• Introducing normalized SSCD for evaluation metrics