🤖 AI Summary
This work challenges the prevailing reliance on monolithic architectures—particularly the Transformer—in contemporary AI systems, which overlooks neuroscientific evidence that diverse cognitive functions emerge from heterogeneous, interacting brain regions. The study systematically argues that the Transformer more accurately models hippocampal function rather than serving as a universal cortical analog. Building on this insight and inspired by the structural and functional heterogeneity of the cerebral cortex, the authors propose a modular, heterogeneous network architecture wherein each module embodies a distinct inductive bias and communicates with others through standardized interfaces. Integrating principles from cytoarchitectonics, functionalism, and modular design, this approach establishes a novel paradigm for AI architecture that enhances generalization while reducing data dependency, thereby reclaiming the benefits of efficient, biologically informed inductive biases.
📝 Abstract
AI researchers describe state-of-the-art models as one thing repeated at scale: the Transformer, wired identically for text, pixels, or speech. Neuroscientists describe the cortex as a mosaic - dense Layer 4 in visual cortex for spatial encoding, thick Layers 5/6 in motion cortex for temporal integration - different jobs solved by different structures. This paper argues the gap is a structural error, not a stylistic one, and is measurable. A century of cytoarchitecture, from Brodmann to single-cell Patch-seq, shows distinct cognitive functions are implemented by qualitatively different structures, not by rescaling one template. The convolutional neural network is the field's own proof: local receptive fields and hierarchical depth encoded this prior directly, reaching strong image recognition on far less data than later architectures needed. The paper traces how this lesson was discarded: the "Hardware Lottery" made the Transformer the path of least resistance, not the principled choice, and Mixture-of-Experts, often cited as diversity, in fact partitions parameters among identical experts. A functionalist analysis shows the Transformer is best understood as a functional analog of the hippocampal formation, not a general-purpose cortex - the same mistake as treating cortex as one giant Broca's area, except the field has now standardized on a giant hippocampus, applied to tasks it was never built for: audition, executive gating, working memory. The paper closes with an alternative: a Heterogeneous Topological Network, a System of Systems in which distinct modules keep the inductive bias their computation demands and communicate through standardized interfaces. This is a design discipline for AI architects, not cognitive science: specify modularity before training, using structural evidence as a design input rather than reverse-engineering architecture from a trained model's behavior.