design neural architectures

Spike Neural Network Architecture Search (SNNaS) faces fundamental challenges including high training complexity and difficulties in co-modeling software and hardware. This paper presents the first systematic survey of recent SNNaS advances from a hardware–software co-design perspective. It analyzes the limitations of conventional Neural Architecture Search (NAS) methods when applied to SNNs, and reveals intrinsic differences between SNNs and Artificial Neural Networks (ANNs) in training dynamics, temporal behavior, and hardware constraints—such as latency, power consumption, and memory bandwidth. To address these, we integrate spike dynamics modeling, differentiable and reinforcement-based NAS algorithms, and hardware performance evaluation frameworks, proposing a constraint-aware search paradigm tailored for edge computing and IoT applications. Our work establishes a unified analytical framework and practical guidelines for SNNaS, advancing the automated design of low-power, high-energy-efficiency neuromorphic computing systems.

Neural architecture design often relies on heuristic rules or expensive search strategies, lacking a principled, differentiable mapping from performance to structure. Method: This paper proposes an automatic architecture optimization framework grounded in structure–performance mapping modeling. We introduce the Architecture Synthesis Neural Network (ASNN), the first model that takes a performance distribution (e.g., accuracy distribution) as input and differentiably synthesizes high-performing architectural parameters—enabling invertible, generalizable mapping from performance to structure. Leveraging a TensorFlow-based multi-layer network performance dataset, ASNN learns this inverse mapping via neural regression and incorporates an iterative prediction mechanism for progressive refinement. Contribution/Results: On both two- and three-layer networks, ASNN discovers novel architectures surpassing the original dataset’s best-performing models, achieving statistically significant average test accuracy improvements. Experiments validate its effectiveness in architecture recommendation, cross-architecture generalization, and iterative optimization.

Formal verification of software architecture remains impractical in industry due to prohibitively high modeling costs. Method: This paper proposes Neural Architecture Inference—a novel approach that automatically learns structured, verifiable architectural models from source code or runtime traces. It establishes the first neuro-symbolic paradigm for architecture inference, integrating graph neural networks and sequence modeling with formal specification languages (e.g., TLA+, Alloy) to close the learning–verification loop. Contribution/Results: We define a six-dimensional research roadmap and present a framework for automated generation of interpretable, formally verifiable architecture models. Experiments demonstrate substantial reduction in modeling effort and enable symbolic verification of architectural constraints—including layer isolation and communication protocols—thereby providing both theoretical foundations and practical pathways for industrial-scale architecture governance.

Existing neural networks are constrained by hierarchical tree-like architectures, which preclude direct communication among sibling nodes and prohibit backward signal propagation to higher-level modules—resulting in weak inter-module collaboration and inefficient representation learning. To address these limitations, we propose the Synchronous Graph Neural Architecture (SGNA), organizing neural units into a modular, dynamically collaborative graph structure that enables arbitrary node-to-node communication and cross-layer signal transmission. Our key contributions are threefold: (1) introducing the first modular graph-structured paradigm for neural architecture design; (2) developing a systematic regularization framework to enforce module independence and load balancing; and (3) generalizing neural architecture search (NAS) to the space of directed acyclic graphs (DAGs). Extensive multi-task experiments demonstrate that SGNA significantly outperforms deep stacked baselines under parameter constraints, achieving superior collaborative representation capability and more comprehensive coverage of the search space.

To address the prohibitively high computational cost of training candidate architectures in neural architecture search (NAS), this paper proposes NASGraph—a zero-cost, data-agnostic lightweight method. Its core innovation lies in modeling neural networks as graphs and, for the first time, employing a graph topological feature—average degree—as a performance proxy, thereby eliminating both training and data dependencies. On NAS-Bench-201, NASGraph identifies the optimal architecture among 200 randomly sampled candidates in just 217 seconds with high accuracy. It further achieves competitive ranking correlation across multiple benchmarks, including NAS-Bench-101, NDS, and Micro TransNAS-Bench-101. By decoupling architecture evaluation from training and data, NASGraph significantly improves evaluation efficiency and cross-benchmark transferability, offering a scalable and practical solution for resource-constrained NAS.

This study addresses the lack of general design principles linking neural network architecture to computational capacity. By systematically evaluating the computational performance of recurrent neural networks with diverse connectivity patterns on Boolean function tasks through large-scale sampling, the work reveals— for the first time—that local 2-cycles and 3-cycles are critical structural motifs for enhancing computational power. It further demonstrates that introducing a small number of sparse connections and biologically inspired interneuron-like units significantly boosts the performance of large-scale networks. The authors construct a comprehensive performance map linking small-network architectures to Boolean function realization, showing that networks containing short cycles achieve optimal performance. Moreover, network performance can be accurately predicted from structural statistics, offering a theoretical foundation and biologically inspired guidance for future neural architecture design.

This work addresses the diminished understanding of neural network fundamentals caused by the widespread use of high-level deep learning libraries. To bridge this gap, the authors construct a complete neural network framework from scratch, eschewing automatic differentiation and prebuilt modules. The implementation explicitly details forward and backward propagation, incorporates multiple activation functions, L2 regularization, and advanced optimizers such as Adam. Designed to balance pedagogical clarity with engineering scalability, the framework demonstrates numerical stability, correctness, and generalization capability on multiclass classification tasks. It thus provides a reproducible and extensible tool for both research and instruction, fostering deeper insight into the core principles of deep learning.

Deploying neural networks on unconventional hardware requires balancing accuracy, energy consumption, and platform-specific physical non-idealities, yet existing neural architecture search (NAS) methods are often confined to specific hardware platforms, lacking cross-platform generalization and fair comparability. To address this, this work proposes UH-NAS, a hardware-agnostic framework that, for the first time, leverages large language models as evolutionary operators within a co-design pipeline integrating pluggable hardware backends, platform-level energy models, and non-ideality simulators. Experiments demonstrate that UH-NAS discovers more diverse and robust architectures on unconventional substrates such as optical Mach–Zehnder interferometer (MZI) arrays, significantly outperforming both conventional and LLM-driven NAS baselines. Ablation studies further confirm the critical roles of system prompting and hardware-software co-design in achieving these gains.

This work proposes a novel paradigm that synergizes large language models (LLMs) with neural architecture search (NAS) to overcome the limited generalizability of conventional NAS methods, which rely on handcrafted search spaces. The approach begins by leveraging an LLM to generate high-quality initial architectures, which are then transformed into “slot-based” architectures containing replaceable modular components. This enables the automatic construction of task-adaptive, structured search spaces that balance open-ended generation with efficient search. Implemented through a modular three-stage pipeline without any human intervention, the method achieves state-of-the-art performance on 11 out of 17 cross-modal tasks, significantly outperforming existing baselines and expert-designed architectures, thereby demonstrating its generality and effectiveness.

This work proposes a hardware-aware, extensible neural architecture search (NAS) framework that decouples the search, optimization, and deployment pipelines to overcome the tight coupling prevalent in existing NAS methods, which hinders adaptation to new hardware or custom operators. The framework employs YAML for unified search space specification and leverages Optuna for efficient architecture search. It supports hardware-in-the-loop evaluation by integrating Docker-based cross-compilation, automated on-device binary generation, and multidimensional performance metrics—including FLOPs, parameter count, and latency—thereby significantly enhancing support for heterogeneous accelerator platforms and reducing engineering overhead in embedded AI deployment.