EvoTreeNAD: Genealogy-Guided Evolution for LLM-Driven Neural Architecture Discovery

2026-09-24

AI Summary

This study addresses the challenges of accumulating iterative progress and high evaluation costs in LLM-driven neural architecture discovery by proposing a zero-seed search framework based on lineage evolution. The method constructs a persistent, seedless lineage tree that leverages offspring percentile rankings to guide evolutionary trajectories, while coordinating Idea and Code agents to automate architecture evolution and evaluation. A theoretical analysis establishes the sustained probabilistic advantage of this mechanism in generating high-reward architectures. Experiments demonstrate that the proposed approach significantly reduces test error on CIFAR and MedMNIST benchmarks, comprehensively outperforming existing NAS/NAD baselines and multiple control strategies.

Abstract

AI-driven scientific discovery accelerates research by autonomously developing solutions and designs. Large language model (LLM) agents support this process through iterative generation and evaluation. Yet these iterations alone do not ensure cumulative progress or establish which directions to pursue next. Costly evaluation further constrains the scope of exploration. Neural architecture discovery brings these challenges together, coupling open-ended design with resource-intensive experimentation. We introduce EvoTreeNAD, a genealogy-guided evolutionary algorithm that constructs trainable architectures without a supplied seed or a hand-specified search space. Starting from an empty root, it grows a persistent genealogy in which each new node represents a complete architecture. Top-percentile values computed from each node and its descendants guide lineage selection. Using the selected design history, an Idea Agent proposes a variant and a Code Agent implements it. Each evaluated variant becomes a child node, expanding the genealogy while providing evidence for subsequent lineage selection. Our theoretical analysis establishes the existence of stationary variation regimes as the genealogy grows. Under specified variation assumptions, sustained top-percentile family values quantify the probability of generating high-reward architectures in these regimes. EvoTreeNAD discovers architectures that outperform the compared NAS and NAD baselines, achieving CIFAR-10/100 test errors of $2.05{\pm}0.06\%$ and $15.09{\pm}0.22\%$. On all six MedMNIST-v2 tasks, the discovered architectures surpass the strongest listed baselines. A controlled CIFAR-10 study further shows that EvoTreeNAD outperforms direct generation, best-of-$N$ greedy continuation, and full-family-mean routing.

Problem

• Neural Architecture Discovery
• Large Language Models
• Open-ended Design
• Costly Evaluation
• Cumulative Progress

Innovation

• Neural Architecture Discovery
• Genealogy-Guided Evolution
• Large Language Model Agents
• Evolutionary Algorithm
• Stationary Variation Regimes