LLM-Guided Neural Architecture Search for Robust Co-Design of Physical Neural Networks

2026-06-08

AI Summary

Deploying neural networks on unconventional hardware requires balancing accuracy, energy consumption, and platform-specific physical non-idealities, yet existing neural architecture search (NAS) methods are often confined to specific hardware platforms, lacking cross-platform generalization and fair comparability. To address this, this work proposes UH-NAS, a hardware-agnostic framework that, for the first time, leverages large language models as evolutionary operators within a co-design pipeline integrating pluggable hardware backends, platform-level energy models, and non-ideality simulators. Experiments demonstrate that UH-NAS discovers more diverse and robust architectures on unconventional substrates such as optical Mach–Zehnder interferometer (MZI) arrays, significantly outperforming both conventional and LLM-driven NAS baselines. Ablation studies further confirm the critical roles of system prompting and hardware-software co-design in achieving these gains.

Abstract

Deploying neural networks on unconventional hardware demands architectures that co-optimize task accuracy and platform-specific constraints such as energy cost, physical non-idealities, and numerical precision. Existing neural architecture search (NAS) methods are typically tailored to a single hardware family, limiting cross-platform comparison and generalization. We introduce Unconventional Hardware Neural Architecture Search (UH-NAS), a hardware-agnostic, LLM-guided NAS framework that integrates language models as evolutionary operators to co-optimize accuracy and inference energy. By exposing hardware as a swappable backend with per-platform energy models, physical constraints, and non-ideality simulators, UH-NAS enables fair system-level comparisons across various backends without modifying the search algorithm. Tested on optical MZI hardware, UH-NAS discovers more diverse, robust architectures than conventional baselines while outperforming existing LLM-to-NAS approaches. Additional ablations on architecture robustness under non-idealities and the role of system prompts highlight the importance of architecture-hardware co-design for emerging computing platforms.

Problem

• neural architecture search
• unconventional hardware
• hardware-aware optimization
• physical non-idealities
• cross-platform generalization

Innovation

• LLM-guided NAS
• hardware-agnostic co-design
• physical neural networks
• non-ideality robustness
• energy-aware architecture search