Provably Reduced Sample Cost in Prior-Guided Hyperparameter Optimization

2026-06-03

AI Summary

This work addresses the high computational cost of large-scale hyperparameter optimization and the lack of theoretical understanding regarding how prior information quantitatively reduces sample complexity. Within the fixed-budget best-arm identification framework, the authors model a prior distribution over the mean performance of configurations and, for the first time, establish a prior-dependent sample complexity bound for multi-fidelity hyperparameter optimization. They further derive an explicit error bound that characterizes the trade-off between the informativeness of the prior and the evaluation budget. Theoretical analysis reveals that highly informative priors can substantially reduce the number of required evaluations. Empirical validation on synthetic benchmarks and LCBench demonstrates the method’s efficacy, achieving up to 90% reduction in evaluation budget while maintaining solution quality, thereby offering theoretical support for green AutoML.

Abstract

Large-scale hyperparameter optimization (HPO) in automated machine learning (AutoML) consumes substantial computational resources, raising growing concerns about scalability and energy efficiency. Existing methods use prior information heuristically to accelerate both black-box and multi-fidelity settings, but they lack a characterization of how prior informativeness quantitatively reduces sample complexity. In this work, we provide the first distribution-dependent sample complexity bounds for multi-fidelity HPO with priors through the formal lens of fixed-budget best-arm identification. By modeling priors directly over arm means as configuration performance, we derive explicit, distribution-dependent error bounds that quantify the relationship between priors and evaluation budget. Our analysis shows that informative priors, which concentrate probability mass on near-optimal arms, yield reductions in the number of required evaluations, whereas baseline performance is recovered with uninformative or misleading priors. We conduct proof-of-concept experiments on a synthetic benchmark and on LCBench, a common multi-fidelity HPO benchmark for deep learning, to confirm our theoretical results, achieving up to 90% budget reduction while retaining solution quality. Together, our results provide a principled foundation for prior-guided and compute-efficient green AutoML.

Problem

• hyperparameter optimization
• sample complexity
• prior information
• multi-fidelity
• AutoML

Innovation

• prior-guided hyperparameter optimization
• sample complexity
• multi-fidelity optimization
• best-arm identification
• green AutoML