Exploiting network topology in brain-scale simulations of spiking neural networks

2026-02-26

AI Summary

This study addresses a critical performance bottleneck in distributed large-scale spiking neural network simulations: global synchronization incurs substantial communication overhead and execution time variability due to waiting for the slowest node. Challenging the conventional view that bandwidth limitations dominate, this work identifies computational time variability as the key constraint. Inspired by the modular architecture of the mammalian brain, the authors propose a hybrid local-global communication framework that maps neuroanatomical regions onto compute nodes to reduce global synchronization frequency. By integrating statistical modeling of computation time distributions with topology-aware task partitioning, they optimize MPI communication patterns. Experimental results on realistic large-scale simulations demonstrate significant performance improvements, offering an energy-efficient pathway for conventional supercomputers and establishing a new benchmark for neuromorphic system design.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Hardware-aware ML
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Social Networks and Social Media: Computational social science
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Simulation code for conventional supercomputers serves as a reference for neuromorphic computing systems. The present bottleneck of distributed large-scale spiking neuronal network simulations is the communication between compute nodes. Communication speed seems limited by the interconnect between the nodes and the software library orchestrating the data transfer. Profiling reveals, however, that the variability of the time required by the compute nodes between communication calls is large. The bottleneck is in fact the waiting time for the slowest node. A statistical model explains total simulation time on the basis of the distribution of computation times between communication calls. A fundamental cure is to avoid communication calls because this requires fewer synchronizations and reduces the variability of computation times across compute nodes. The organization of the mammalian brain into areas lends itself to such an optimization strategy. Connections between neurons within an area have short delays, but the delays of the long-range connections across areas are an order of magnitude longer. This suggests a structure-aware mapping of areas to compute nodes allowing for a partition into more frequent communication between nodes simulating a particular area and less frequent global communication. We demonstrate a substantial performance gain on a real-world example. This work proposes a local-global hybrid communication architecture for large-scale neuronal network simulations as a first step in mapping the structure of the brain to the structure of a supercomputer. It challenges the long-standing belief that the bottleneck of simulation is synchronization inherent in the collective calls of standard communication libraries. We provide guidelines for the energy efficient simulation of neuronal networks on conventional computing systems and raise the bar for neuromorphic systems.

Problem

• spiking neural networks
• large-scale simulation
• communication bottleneck
• network topology
• distributed computing

Innovation

• brain-inspired computing
• spiking neural networks
• structure-aware mapping
• hybrid communication architecture
• large-scale simulation