🤖 AI Summary
This work addresses the inefficiency of conventional black-box optimization in hierarchical heterogeneous search spaces where a large fraction of configurations are infeasible due to crash risks. To tackle this challenge, the authors propose a feasibility-first exploration strategy combined with a novel Thermal Budget Annealing (TBA) method that decomposes optimization into distinct exploration and exploitation phases. Robustness is further enhanced through trial timeouts, subspace blacklisting, and Tree-structured Parzen Estimators (TPE). The study introduces DeployBench, a new benchmark featuring hidden crash zones and non-uniform evaluation costs. Experimental results demonstrate that the proposed approach significantly improves the success rate of discovering valid model families under constraints and substantially reduces the overhead of invalid evaluations in both synthetic and real-world GPU deployment tasks.
📝 Abstract
Deploying machine learning models under production constraints requires joint optimization over model family, quantization scheme, runtime backend, and serving configuration. This induces a hierarchical mixed-variable search space in which many configurations are invalid: evaluations may crash, exceed memory limits, or violate latency constraints. Standard black-box optimizers such as Tree-structured Parzen Estimators (TPE) and constrained Bayesian optimization are effective when valid configurations are common, but they can spend a large fraction of a small evaluation budget on invalid or uninformative trials in hostile deployment spaces. This paper studies that regime and asks whether optimization should be decomposed into an explicit exploration stage followed by model-guided exploitation. We propose Thermal Budget Annealing (TBA), a feasible-first exploration procedure that maps valid and feasible regions before warm-starting TPE. The method includes two robustness mechanisms for hostile hardware: trial timeouts that abort clearly infeasible evaluations early, and subspace blacklisting that temporarily suppresses categorical subspaces after repeated failures. We also introduce DeployBench, a benchmark suite for deployment optimization with hierarchical structure, hidden crash zones, hard constraints, and unequal evaluation costs. On synthetic benchmarks and real GPU deployment with five pre-trained vision models across five GPU targets (NVIDIA H100, A100, RTX 5080, L4, and T4), the proposed hybrid improves model-family discovery under tight constraints while reducing wasted budget relative to cold-start TPE.