🤖 AI Summary
This work addresses the challenge of efficiently predicting the accuracy gains of Best-of-N inference without performing full-scale sampling. The authors propose a lightweight prediction method that leverages only three core statistical features derived from a single model pass on the validation set: prompt-level consistency distribution, position of the first correct sample, and variance in generation length, supplemented by entropy. A ridge regression-based predictor is constructed and theoretically grounded through Bootstrap-Lasso stability analysis and concentration bounds on linear approximation residuals. Evaluated across diverse models, training strategies, and tasks, the approach achieves a Spearman correlation of ρ=0.90 with actual Best-of-N performance gains, substantially reducing the computational cost associated with reward model evaluation.
📝 Abstract
Best-of-$N$ inference scaling (drawing $N$ candidate answers from a language model and returning the one a reward model ranks highest) improves accuracy by an amount that varies across models, but predicting that amount in advance currently requires running the procedure end-to-end. Prior work links cheap statistics of a model's sampled outputs and validation-set correctness (how often samples agree, how diverse they are, how confident the model is, and where correct samples appear) to model behavior, but does not isolate which of these form a stable, compact predictor of best-of-$N$ gain. We fit ridge predictors on features computed from a single labeled validation-set sampling pass, use bootstrap-Lasso as a stability analysis of the candidate feature set, and give a concentration analysis with an explicit linear-approximation residual. Across three base-model families, six post-training methods, and math and reasoning task domains, the stability analysis identifies a strict three-feature core spanning prompt-level agreement spread, label-assisted first-correct-sample position, and completion-length variance; a compact ridge predictor built from this core plus an entropy add-on reaches Spearman $ρ= 0.90$ with actual best-of-$N$ gain under a reward-model verifier. The intended use is labeled validation-set screening of candidate configurations before paying the full reward-model scoring cost.