Pool-Select-Refine: Allocation-Aware Generative Dataset Distillation with Soft-Label-Guided Latent Refinement

2026-06-01

AI Summary

This work addresses the limitations of existing diffusion-based image dataset distillation methods, which adopt a “generate-as-you-use” strategy that leads to rigid sample allocation and risks either information redundancy or insufficiency. To overcome this, the authors propose a two-stage distillation framework: first, an over-complete pool of candidate samples is generated and pruned according to a predefined budget; then, the selected samples undergo semantic-aware refinement in latent space, guided by soft labels from a teacher model. This approach uniquely decouples generation, selection, and refinement, introducing a budget-aware selection mechanism and soft-label-driven latent-space optimization. Experiments demonstrate that the method significantly outperforms current diffusion-based distillation techniques on both large-scale and fine-grained image classification benchmarks, confirming its effectiveness and generalizability.

Abstract

Diffusion-based dataset distillation has recently emerged as a promising paradigm for condensing large-scale datasets into compact synthetic sets. By leveraging pretrained generative priors, these methods can produce realistic class-conditional samples more efficiently than traditional matching-based approaches. However, most existing diffusion-based methods still adopt a rigid ``Generate-and-Use'' strategy, where the generated samples are directly treated as the final distilled set under a fixed images-per-class budget. Such a design tightly couples candidate generation with final budget allocation, which may result in redundant waste of the limited budget or insufficiently informative samples. In this paper, we propose ``Pool-Select-Refine'', a two-stage framework for allocation-aware generative dataset distillation. First, instead of directly using a fixed number of generated samples, we construct an over-complete candidate pool and select a compact subset under the target budget. Second, we refine the selected samples in latent space using soft-label supervision derived from the teacher model, improving semantic alignment while preserving the generative prior. This design explicitly decouples generation, selection, and refinement, enabling more effective use of the distillation budget. Experiments on large-scale and fine-grained image classification benchmarks show that the proposed framework delivers consistent gains over diffusion-based baselines. The results suggest that introducing a curation stage before refinement is a simple yet effective way to improve diffusion-based dataset distillation.

Problem

• dataset distillation
• diffusion models
• budget allocation
• generative prior
• soft-label supervision

Innovation

• dataset distillation
• diffusion models
• allocation-aware
• soft-label refinement
• latent space optimization