🤖 AI Summary
Deep learning training suffers from performance instability and unreliable hyperparameter selection due to high sensitivity to multi-factor interactions. To address this, we propose a risk-aware optimization framework integrating experimental design with Shapley value decomposition. Our method introduces a novel dual-path Shapley estimation: a plug-in estimator based on conditional means ensures interpretability, while a least-squares reconstruction path enables bias correction and cross-factor comparability in heterogeneous parameter spaces; we theoretically derive an upper bound on the optimality gap. The framework integrates standardized Shapley estimation, uncertainty quantification, and lightweight search to efficiently identify robust configurations under budget constraints. Extensive evaluation across multiple datasets and experimental design scenarios demonstrates significant improvements in optimal configuration identification accuracy and rank preservation, effectively reducing decision risk while balancing performance gains and model interpretability.
📝 Abstract
We propose FACTORS, a framework that combines design of experiments with Shapley decomposition to address performance and stability issues that are sensitive to combinations of training factors. Our approach consistently estimates main effects and two-factor interactions, then integrates them into a risk-adjusted objective function that jointly accounts for uncertainty and cost, enabling reliable selection of configurations under a fixed budget. Effect estimation is implemented through two complementary paths: a plug-in path based on conditional means, and a least-squares path that reconstructs Shapley contributions from samples. These paths are designed to work complementarily even when design density and bias levels differ. By incorporating standardization of estimates, bias correction, and uncertainty quantification, our procedure ensures comparability across heterogeneous factor spaces and designs, while a lightweight search routine yields configurations within practical time even for large factor spaces. On the theoretical side, we provide error decompositions, sample complexity analysis, and upper bounds on optimality gaps. On the interpretive side, we summarize main effects and interactions in map form, highlighting adjustment priorities and safe improvement pathways. Across diverse datasets and design conditions, our approach improves rank preservation and optimal configuration identification, reduces decision-making risks, and offers a tuning foundation that delivers interpretable justification alongside stable performance gains even under budget constraints.