HARP: Efficient Data Selection for Finetuning Large Language Models

2026-06-05

AI Summary

This work addresses the critical challenge of efficiently selecting a subset of training data that enhances downstream performance while avoiding redundant computation in large language model fine-tuning. The authors propose a training-informed data selection method that constructs a node-to-leaf hierarchical data structure and leverages gradient-guided utility estimation combined with empirical Bayes posterior inference to approximate the value of unseen samples. A novel dual-strategy selection mechanism is introduced: HARP-C employs hierarchical pruning to control redundancy, while HARP-E utilizes a dual-envelope reward scheme to capture complementary regions of high utility. Evaluated on standard benchmarks, the approach achieves up to an 8.9-point improvement over the strongest baseline using only approximately one-seventh of the original training data, delivering superior performance at substantially reduced evaluation cost.

Abstract

Finetuning data selection requires balancing two competing goals: selecting examples that improve the downstream objective, and doing so without repeatedly finetuning models. Train-free selectors are scalable but rely on proxies such as embedding similarity or clustering, which may not match the target objective. Train-based selectors better reflect downstream utility through gradient signals, subset evaluation, or Shapley attribution, but require many costly train--evaluate iterations. We propose Hierarchical Active Region Pruning (HARP), an efficient train-based selector that preserves downstream alignment while reducing selection cost. HARP organizes the training pool into a node--leaf hierarchy, evaluates only representative leaves, and infers unmeasured utilities with empirical Bayes posteriors. It then selects data using two complementary envelopes: HARP-C, which conservatively controls redundancy, and HARP-E, which additively rewards complementary regions. We theoretically show that, under local smoothness and bounded estimation error, HARP controls selection error while reducing train--evaluate cost. We further validate that HARP variants achieve the best result and outperform the strongest baseline by up to $+8.9$ points, while using roughly $7\times$ fewer training examples.

Problem

• data selection
• finetuning
• large language models
• train-based selection
• selection efficiency

Innovation

• data selection
• efficient finetuning
• hierarchical pruning
• empirical Bayes
• large language models