TopoNets: High Performing Vision and Language Models with Brain-Like Topography

2025-01-27

AI Summary

This study addresses the lack of brain-like spatial functional organization in current AI models, which hinders simultaneous optimization of computational efficiency and task performance. We propose TopoLoss—a novel loss function enabling plug-and-play topological awareness during training—achieving, for the first time, general-purpose topological representation learning without accuracy degradation. The method is architecture-agnostic, integrating seamlessly with ResNet, ViT, GPT-Neo, and NanoGPT to form TopoNets. On ImageNet and WikiText, TopoNets retain state-of-the-art (SOTA) performance. Critically, they reproduce key topological signatures observed in human visual and language cortices—including retinotopic and syntacto-topic organization—thereby enhancing local processing capability, low-dimensional encoding efficiency, and cross-modal neural response prediction accuracy. This work establishes a new paradigm for developing multimodal AI models that simultaneously achieve high accuracy, strong biological plausibility, and improved interpretability.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: (Large) Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Neurons in the brain are organized such that nearby cells tend to share similar functions. AI models lack this organization, and past efforts to introduce topography have often led to trade-offs between topography and task performance. In this work, we present TopoLoss, a new loss function that promotes spatially organized topographic representations in AI models without significantly sacrificing task performance. TopoLoss is highly adaptable and can be seamlessly integrated into the training of leading model architectures. We validate our method on both vision (ResNet-18, ResNet-50, ViT) and language models (GPT-Neo-125M, NanoGPT), collectively TopoNets. TopoNets are the highest-performing supervised topographic models to date, exhibiting brain-like properties such as localized feature processing, lower dimensionality, and increased efficiency. TopoNets also predict responses in the brain and replicate the key topographic signatures observed in the brain's visual and language cortices. Together, this work establishes a robust and generalizable framework for integrating topography into leading model architectures, advancing the development of high-performing models that more closely emulate the computational strategies of the human brain.

Problem

• AI efficiency
• neural grouping
• human-like computation

Innovation

• TopoLoss
• Efficiency Enhancement
• Neural Pattern Replication