AgenticRS-EnsNAS: Ensemble-Decoupled Self-Evolving Architecture Search

2026-03-20

AI Summary

This work addresses the prohibitive evaluation cost in industrial neural architecture search (NAS), where each candidate architecture typically requires validation via ensembles of 50–200 models, incurring an O(M) computational overhead per evaluation. To overcome this bottleneck, we propose a decoupled ensemble-based NAS framework that leverages a lightweight dual-learner system to estimate key ensemble statistics, enabling accurate prediction of full-ensemble performance under theoretical guarantees. This reduces the per-architecture validation cost to O(1), with full ensemble deployment reserved only for top-performing candidates. The framework reveals an orthogonal optimization mechanism between base model diversity and accuracy gains, and unifies support for both continuous and discrete architecture spaces—encompassing closed-form optimization, constrained differentiable search, and LLM-guided monotonic acceptance strategies. Moreover, we rigorously establish sufficient conditions under homogeneity assumptions for monotonic reduction of ensemble error, substantially alleviating the validation burden in industrial-scale NAS.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Ensemble Methods
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Neural Architecture Search (NAS) deployment in industrial production systems faces a fundamental validation bottleneck: verifying a single candidate architecture pi requires evaluating the deployed ensemble of M models, incurring prohibitive O(M) computational cost per candidate. This cost barrier severely limits architecture iteration frequency in real-world applications where ensembles (M=50-200) are standard for robustness. This work introduces Ensemble-Decoupled Architecture Search, a framework that leverages ensemble theory to predict system-level performance from single-learner evaluation. We establish the Ensemble-Decoupled Theory with a sufficient condition for monotonic ensemble improvement under homogeneity assumptions: a candidate architecture pi yields lower ensemble error than the current baseline if rho(pi) < rho(pi_old) - (M / (M - 1)) * (Delta E(pi) / sigma^2(pi)), where Delta E, rho, and sigma^2 are estimable from lightweight dual-learner training. This decouples architecture search from full ensemble training, reducing per-candidate search cost from O(M) to O(1) while maintaining O(M) deployment cost only for validated winners. We unify solution strategies across pipeline continuity: (1) closed-form optimization for tractable continuous pi (exemplified by feature bagging in CTR prediction), (2) constrained differentiable optimization for intractable continuous pi, and (3) LLM-driven search with iterative monotonic acceptance for discrete pi. The framework reveals two orthogonal improvement mechanisms -- base diversity gain and accuracy gain -- providing actionable design principles for industrial-scale NAS. All theoretical derivations are rigorous with detailed proofs deferred to the appendix. Comprehensive empirical validation will be included in the journal extension of this work.

Problem

• Neural Architecture Search
• Ensemble Learning
• Computational Cost
• Validation Bottleneck
• Industrial Deployment

Innovation

• Ensemble-Decoupled Architecture Search
• Neural Architecture Search
• Computational Efficiency
• Ensemble Theory
• LLM-driven Search