Beyond Background Bias: Saliency-Driven Prototype Alignment for Dataset Distillation

2026-07-28

AI Summary

Existing diffusion-based dataset distillation methods struggle to balance structural consistency and generalization due to weak alignment between latent prototypes and class-discriminative regions, as well as susceptibility to background interference. This work proposes a saliency-driven two-stage prototype alignment framework that operates without fine-tuning the frozen diffusion backbone (e.g., LDM or DiT). By integrating Grad-CAM to generate high-confidence discriminative regions and introducing a hard prototype refinement strategy to enhance prototype diversity and discriminability, the method leverages only a lightweight classifier to achieve significant improvements over strong baselines across multiple benchmarks. The approach effectively boosts the representativeness, training efficacy, and generalization capability of synthesized data.

Abstract

Dataset distillation aims to synthesize compact datasets that can approximate the performance of full-data training while significantly reducing computational and storage costs. However, diffusion-based distillation methods often struggle to preserve structural coherence and generalization, especially in visually complex domains. This issue often stems from latent prototypes that are weakly aligned with class-discriminative regions and contaminated by irrelevant background, thereby degrading generation quality and generalization. To address this limitation, we propose a saliency-driven distillation framework that constructs class-discriminative latent prototypes to enhance representativeness and generalization. The framework proceeds in two stages: (1) ensemble Grad-CAM saliency is used to construct prototypes emphasizing high-confidence regions, and (2) hard prototype refinement is then applied to construct challenging yet class-consistent prototypes, thereby enhancing discriminability and diversity. Importantly, the diffusion backbones (e.g., LDM and DiT) remain frozen; only lightweight classifiers used for saliency extraction are trained. Extensive experiments across multiple benchmarks demonstrate consistent performance improvements over strong baselines. Code will be released.

Problem

• dataset distillation
• diffusion models
• prototype alignment
• saliency
• background bias

Innovation

• saliency-driven
• prototype alignment
• dataset distillation
• Grad-CAM
• diffusion models