neural architecture design

Current monolithic perception models fall short in interpretability, compositional generalization, and adaptive robustness, limiting their ability to emulate human cognition. Inspired by the modular architecture of the cerebral cortex, predictive processing mechanisms, and principles of cross-modal integration, this work proposes a hierarchical perception framework composed of specialized interactive modules. The architecture achieves human-like perception through explicit reasoning, hierarchical predictive feedback loops, and a shared latent space. This study represents the first systematic effort to incorporate cortical modularity theory into the design of artificial intelligence perception systems. Empirical results demonstrate that the proposed architecture significantly enhances representational stability, system transparency, and alignment with human cognitive patterns, all while maintaining competitive performance.

Current visual systems exhibit insufficient robustness to noise, deformation, and out-of-distribution (OOD) data, and lack unsupervised, compositional, and interpretable neural representation mechanisms. To address these limitations, we propose the Collaborative Network Architecture (CNA), which introduces a novel “network fragment” dynamic composition mechanism. CNA unsupervisedly discovers local structural primitives via statistical learning and recursively assembles them through structured sparse connectivity, yielding global–local coupled representations that adaptively encode sensory patterns. Without any labeled data, CNA achieves robust recognition under noise and geometric deformation, zero-shot schema completion, and generalization to unseen patterns—significantly enhancing OOD generalization. Its modular, interpretable neural representations establish a new paradigm for invariant object recognition, bridging compositional structure with biological plausibility and computational efficiency.

Hierarchical concept identification in brain network modeling is hindered by noise interference, neuronal failure, and sparse connectivity. Method: This paper proposes a neuron-level abstraction and divide-and-conquer approach, establishing—for the first time—a rigorous formal refinement relationship between multi-neuronal networks and single-neuron abstract networks. The method integrates abstract interpretation, neurosymbolic modeling, and hierarchical concept identification theory, reducing high- and low-connectivity multi-neuronal networks (H/L) into verifiable single-neuron abstract networks (A₁/A₂), while preserving semantic consistency via the refinement relation. Contribution/Results: Experiments demonstrate that the framework significantly improves traceability, verifiability, and scalability of brain network analysis. It provides a formal foundation for hierarchical modeling of complex neural mechanisms, enabling principled abstraction while maintaining fidelity to underlying biological structure and dynamics.

Non-modular neural networks suffer from exponential sample complexity growth with input dimensionality in high-dimensional combinatorial tasks—a fundamental bottleneck for generalization. Method: We theoretically and empirically investigate modular neural networks’ generalization mechanisms. We first establish, for the first time, a rigorous theoretical guarantee that modular architectures achieve dimension-independent sample complexity. Methodologically, we propose a task-intrinsic dimensionality–driven modular architecture design and a theory-guided learning rule. Results: Experiments demonstrate significant improvements over baselines in both in-distribution and out-of-distribution generalization, achieving dimension-agnostic efficient learning on high-dimensional combinatorial tasks—thereby breaking conventional scaling laws. Our core contributions are threefold: (i) establishing the first formal theoretical foundation for modular generalization; (ii) devising a provably optimal, theory-grounded learning mechanism; and (iii) empirically validating its fundamental superiority for high-dimensional combinatorial generalization.

Existing neural networks are constrained by hierarchical tree-like architectures, which preclude direct communication among sibling nodes and prohibit backward signal propagation to higher-level modules—resulting in weak inter-module collaboration and inefficient representation learning. To address these limitations, we propose the Synchronous Graph Neural Architecture (SGNA), organizing neural units into a modular, dynamically collaborative graph structure that enables arbitrary node-to-node communication and cross-layer signal transmission. Our key contributions are threefold: (1) introducing the first modular graph-structured paradigm for neural architecture design; (2) developing a systematic regularization framework to enforce module independence and load balancing; and (3) generalizing neural architecture search (NAS) to the space of directed acyclic graphs (DAGs). Extensive multi-task experiments demonstrate that SGNA significantly outperforms deep stacked baselines under parameter constraints, achieving superior collaborative representation capability and more comprehensive coverage of the search space.

This work addresses the challenge of balancing stability and plasticity in continual learning under sequential data scenarios. Inspired by the modular organization of the human brain, the authors propose MoRe, a novel framework that constructs a theoretically identifiable hierarchical modular structure in representation space. MoRe decomposes knowledge into shared foundational modules and task-specific modules, enabling module reuse, alignment, and expansion. By leveraging temporal delayed dependencies to uncover intrinsic sequence structures and integrating modular learning with identifiability constraints, MoRe achieves structured knowledge organization and protection without requiring explicit task boundaries. Experiments on synthetic benchmarks and activation data from large language models demonstrate that MoRe learns interpretable hierarchical representations and significantly improves the stability-plasticity trade-off in continual learning.

This study addresses the central challenge in continual learning of balancing plasticity and stability while mitigating interference between tasks, particularly when they share network structures. Using a sequential A-B-A paradigm, the authors systematically compare modular recurrent networks against monolithic baselines to investigate how modularity, task similarity, and representational dimensionality interact. They find that the effectiveness of modularity hinges critically on the representational dimensionality induced by weight initialization scale: in low-dimensional “rich” regimes, modular networks adaptively form task-specific subspaces that overlap, partially align, or separate according to task similarity, significantly outperforming monolithic architectures; in high-dimensional “lazy” regimes, both approaches perform comparably. The work frames robustness and safety as problems of adaptive subspace allocation and validates this mechanism through controlled weight scaling and internal geometric analysis.

This work proposes a biologically inspired approach to generate sparse, modular recurrent neural networks from highly compressed genotypic encodings for efficiently solving complex temporal tasks. By integrating genotypic compression with modular reservoir computing for the first time, the method employs a hypernetwork to learn a structured generative process and introduces a dual developmental-evolutionary learning paradigm augmented with curriculum-based meta-learning. The resulting networks exhibit strong modularity and sparsity, achieving high performance and robustness with minimal task-specific training. Empirical evaluations demonstrate that the generated architectures significantly outperform existing methods across multiple challenging temporal benchmarks, highlighting the efficacy of combining developmental principles with evolutionary and meta-learning strategies in neural architecture design.

This work challenges the prevailing reliance on monolithic architectures—particularly the Transformer—in contemporary AI systems, which overlooks neuroscientific evidence that diverse cognitive functions emerge from heterogeneous, interacting brain regions. The study systematically argues that the Transformer more accurately models hippocampal function rather than serving as a universal cortical analog. Building on this insight and inspired by the structural and functional heterogeneity of the cerebral cortex, the authors propose a modular, heterogeneous network architecture wherein each module embodies a distinct inductive bias and communicates with others through standardized interfaces. Integrating principles from cytoarchitectonics, functionalism, and modular design, this approach establishes a novel paradigm for AI architecture that enhances generalization while reducing data dependency, thereby reclaiming the benefits of efficient, biologically informed inductive biases.

This study addresses the limitations of current network science in capturing the emergence mechanisms of collective intelligence in human–AI hybrid collectives. Integrating network science, collective cognition, and multi-agent systems theory, the work proposes a heterogeneous network model of human–AI teams through the lenses of attention, memory, and reasoning. It investigates how task environments, network topology, individual cognitive processes, and incentive mechanisms jointly shape collective performance. The research identifies distinctive structural roles unique to hybrid systems—such as humans acting as gatekeepers to AI subnetworks—and reinterprets classic trade-offs like exploration–exploitation and efficiency–redundancy within hybrid contexts. By clarifying which network effects remain robust and which require theoretical revision, this work provides a foundational framework for the organization, governance, and responsible development of hybrid intelligent systems.