GraphIR: Architecture-Level Search States for LLM-Guided Neural Architecture Evolution

2026-08-02

AI Summary

This work addresses the challenge that large language models (LLMs) struggle to accurately infer architectural states—such as tensor dependencies, editable components, and compatibility constraints—from executable code in neural architecture search (NAS). To bridge this semantic gap, the paper introduces GraphIR, a graph-based intermediate representation tailored for architecture evolution. GraphIR provides LLMs with a structured state through three complementary views: computational skeleton, mutation surface, and validity envelope, thereby enabling efficient and guided architectural mutations. This approach constitutes the first mutation-oriented architectural state representation, significantly enhancing LLMs’ capabilities in dependency reasoning and validity assessment. Experiments demonstrate that GraphIR achieves state-of-the-art search performance on the NAS-Dependency benchmark and six downstream tasks, including CLRS, while maintaining model scalability and end-to-end efficiency.

Abstract

Large language models (LLMs) enable neural architecture search (NAS) directly over executable neural network programs. However, code-level flexibility does not provide the architecture state needed for effective mutation: LLMs must infer tensor dependencies, editable components, and compatibility constraints from implementation details. To address this representation mismatch, we propose GraphIR, an architecture-aware intermediate representation that supplements executable programs with a mutation-aligned candidate state. GraphIR organizes each candidate through three complementary views: a computation skeleton describing tensor flow, a mutation surface exposing editable modules and operations, and a validity envelope capturing interface contracts, propagated shapes, and downstream dependencies. To evaluate our method, we construct NAS-Dependency, a 120-question benchmark covering six complementary dependency-reasoning dimensions. The diagnostic shows that GraphIR is particularly effective at identifying exact producer occurrences, tracing dependency propagation, and diagnosing interface and failure risks. Across six downstream benchmarks including CLRS, GraphIR achieves the best overall search performance while maintaining comparable model size and favorable end-to-end NAS efficiency when integrated into OpenEvolve. These results show that a mutation-oriented architecture state provides an effective interface between executable neural programs and LLM-guided architecture evolution.

Problem

• neural architecture search
• large language models
• architecture representation
• mutation
• dependency reasoning

Innovation

• GraphIR
• neural architecture search
• large language models
• intermediate representation
• mutation-aware state