🤖 AI Summary
This work addresses the challenge of preserving representational quality while achieving high storage efficiency in dataset condensation under extremely low-bit quantization. To this end, we propose a plug-and-play compression framework that, for the first time, introduces post-training quantization to this task. The method mitigates accuracy degradation through patch-level local quantization, reduces parameter overhead via quantization-aware clustering, and compensates for quantization-induced distribution shifts with a dedicated alignment module. Notably, our approach requires no additional training and consistently outperforms existing methods across CIFAR-10/100, Tiny ImageNet, and ImageNet subsets. Under extreme 2-bit compression with an image-per-class (IPC) budget of 1, it nearly doubles test accuracy—from 26.0% to 54.1%—demonstrating substantial gains in both efficiency and fidelity.
📝 Abstract
Dataset Condensation (DC) distills knowledge from large datasets into smaller ones, accelerating training and reducing storage requirements. However, despite notable progress, prior methods have largely overlooked the potential of quantization for further reducing storage costs. In this paper, we take the first step to explore post-training quantization in dataset condensation, demonstrating its effectiveness in reducing storage size while maintaining representation quality without requiring expensive training cost. However, we find that at extremely low bit-widths (e.g., 2-bit), conventional quantization leads to substantial degradation in representation quality, negatively impacting the networks trained on these data. To address this, we propose a novel \emph{patch-based post-training quantization} approach that ensures localized quantization with minimal loss of information. To reduce the overhead of quantization parameters, especially for small patch sizes, we employ quantization-aware clustering to identify similar patches and subsequently aggregate them for efficient quantization. Furthermore, we introduce a refinement module to align the distribution between original images and their dequantized counterparts, compensating for quantization errors. Our method is a plug-and-play framework that can be applied to synthetic images generated by various DC methods. Extensive experiments across diverse benchmarks including CIFAR-10/100, Tiny ImageNet, and ImageNet subsets demonstrate that our method consistently outperforms prior works under the same storage constraints. Notably, our method nearly \textbf{doubles the test accuracy} of existing methods at extreme compression regimes (e.g., 26.0\% $\rightarrow$ 54.1\% for DM at IPC=1), while operating directly on 2-bit images without additional distillation.