Distilling Diffusion Score Discrepancy for Efficient Training Data Attribution

2026-09-29

AI Summary

This study addresses the computational expense, retraining requirements, and deviation from true generative behavior inherent in existing training data attribution methods for diffusion models. To overcome these limitations, this work proposes an attribution framework based on local score differences, introducing the first metric that directly targets generative behavior. Furthermore, it presents TIDE, a student model that integrates Kronecker-factored curvature with knowledge distillation to enable efficient estimation. By eliminating the need for retraining, TIDE accurately identifies critical training samples and performs attribution within milliseconds, improving efficiency by several orders of magnitude. Extensive evaluations demonstrate that the proposed approach surpasses state-of-the-art methods across multiple benchmarks, offering a scalable and precise solution for understanding data influence in diffusion-based generation.

Abstract

Training data attribution for diffusion models aims to identify the training samples that influence a generated instance, but existing methods either require costly per-sample gradient computation or query-specific model optimization. Moreover, most methods attribute changes in a proxy loss rather than changes in the actual model's generative behavior. We address these limitations by formulating attribution directly with a local score discrepancy measure, which applies to any diffusion variant (including DDPM, EDM, and flow matching), and by showing that such measure can be estimated without retraining, as a preconditioned gradient similarity. We instantiate this estimator as Training-data Influence via score Discrepancy (TID), which uses Kronecker-factored curvature to avoid random projections and per-sample gradient storage. We then distill TID into TIDE, a forward-only student trained online to reproduce the teacher's rankings from the diffusion model's internal activations. Under counterfactual evaluation on CIFAR-10, ArtBench-10, and MS-COCO, TID matches or outperforms state-of-the-art approaches, while TIDE retains most of TID's accuracy at four to five orders of magnitude lower per-query cost, attributing generated samples in milliseconds and faster than the generation itself.

Problem

• Training data attribution
• Diffusion models
• Score discrepancy
• Computational efficiency

Innovation

• Training Data Attribution
• Diffusion Models
• Score Discrepancy
• Kronecker-factored Curvature
• Knowledge Distillation