Discovering Functionally Selective Brain Regions with a Deep Topographic Multimodal Model

2026-06-08

AI Summary

Existing topological models are predominantly unimodal and exhibit fragmented inter-layer representations, limiting their ability to capture the continuity of cortical processing pathways and cross-modal integration. This work proposes Topo-Omni, a unified cross-modal topological model that embeds visual, auditory, and language/cognitive functions into a single, continuous, brain-inspired cortical manifold. By applying spatially smooth regularization to fine-tune pretrained foundation models, Topo-Omni constructs a deep topological multimodal architecture. It is the first model to achieve integrated multimodal representation within a unified continuous topological space, providing computational support for the “single spatial principle” as a mechanism organizing cross-modal, hierarchical cortical representations. The resulting functional clusters align closely with human neuroimaging data, and computational perturbations successfully replicate known human perceptual biases and lesion effects, while also predicting and validating novel networks for natural scene and animal recognition.

Abstract

Nearby neurons in cortex share similar response profiles, producing systematic spatial organization across sensory and cognitive systems. Recent topographic models reproduce aspects of this structure but remain unimodal and spatially constrain each layer separately, yielding fragmented maps that capture neither the contiguity of cortical processing streams nor their integration across modalities. We introduce Topo-Omni, a topographic multimodal model in which visual, auditory, and language/cognitive processing share a single contiguous in-silico sheet. Built by fine-tuning a pretrained foundation model with a spatial smoothness objective, this architecture develops clusters across modalities that are consistent with human neuroimaging, from sensory to cognitive systems. Driving or suppressing a cluster selectively biases or impairs perception, paralleling human intervention studies. Finally, we use our model to screen for novel clusters in-silico and discover new natural landscape and animal networks which we validate in human data. A single spatial principle thus organizes representations across modalities and processing stages, yielding testable hypotheses about cortical organization.

Problem

• topographic organization
• multimodal integration
• functional selectivity
• cortical continuity
• brain mapping

Innovation

• topographic modeling
• multimodal integration
• spatial smoothness
• functional selectivity
• in-silico discovery