🤖 AI Summary
Existing data pruning methods suffer significant performance degradation under high label noise and struggle to effectively retain informative samples. This work systematically investigates the behavior of pruning strategies in both noisy and noise-free settings, and for the first time explicitly identifies data redundancy, problematic samples, and inter-sample dependencies as three universal factors governing pruning efficacy. Through empirical analysis of two dominant pruning paradigms across standard classification benchmarks and mainstream neural architectures, the study demonstrates the consistent influence of these factors under diverse data distributions and training protocols. The findings not only expose fundamental limitations of current approaches but also offer a new perspective toward designing robust pruning methods.
📝 Abstract
The performance of deep learning models is affected by not only data quantity but also data quality. Data pruning is a process by which practitioners can reduce the size of a dataset by only keeping the most important training data points, thereby achieving similar test set performance. We empirically investigate two popular data pruning methods under noisy and noiseless conditions and show that these methods fail in the presence of significant label noise. We highlight that the success of data pruning is distinctly affected by three factors: redundancy in the dataset, the presence of problematic samples, and interdependence between samples. We perform a detailed investigation on commonly used benchmark classification datasets and neural network architectures. We find that our observations are consistent across data distributions and training protocols.