Neuro-inspired Ensemble-to-Ensemble Communication Primitives for Sparse and Efficient ANNs

2025-08-19

AI Summary

Conventional artificial neural networks (ANNs) lack biologically plausible structural priors, limiting their efficiency and interpretability. Method: Inspired by modular, hierarchical, and sparse inter-neuronal-group connectivity in the mouse visual cortex, we propose G2GNet—the first ANN architecture that explicitly encodes biological functional connectivity as a structural prior. It integrates three core components: (i) neuron-group-level sparse feedforward topology, (ii) dynamic sparse training (DST), and (iii) a Hebbian-like plasticity mechanism for activity-correlation-driven structural rewiring. Contribution/Results: On Fashion-MNIST, CIFAR-10, and CIFAR-100, G2GNet achieves up to 75% weight sparsity while improving accuracy over dense baselines by up to 4.3%. It simultaneously reduces memory footprint and computational cost. This work establishes a novel, scalable paradigm for biologically inspired, efficient sparse ANN design.

Technology Category

Cognitive Modeling & Cognitive Systems: Neural Spike CodingComputer Vision: Generative Adversarial Networks (GANs) for VisionMachine Learning: Deep Neural Architectures and Foundation Models

Application Category

Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsEconomics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystemsSearch and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

The structure of biological neural circuits-modular, hierarchical, and sparsely interconnected-reflects an efficient trade-off between wiring cost, functional specialization, and robustness. These principles offer valuable insights for artificial neural network (ANN) design, especially as networks grow in depth and scale. Sparsity, in particular, has been widely explored for reducing memory and computation, improving speed, and enhancing generalization. Motivated by systems neuroscience findings, we explore how patterns of functional connectivity in the mouse visual cortex-specifically, ensemble-to-ensemble communication, can inform ANN design. We introduce G2GNet, a novel architecture that imposes sparse, modular connectivity across feedforward layers. Despite having significantly fewer parameters than fully connected models, G2GNet achieves superior accuracy on standard vision benchmarks. To our knowledge, this is the first architecture to incorporate biologically observed functional connectivity patterns as a structural bias in ANN design. We complement this static bias with a dynamic sparse training (DST) mechanism that prunes and regrows edges during training. We also propose a Hebbian-inspired rewiring rule based on activation correlations, drawing on principles of biological plasticity. G2GNet achieves up to 75% sparsity while improving accuracy by up to 4.3% on benchmarks, including Fashion-MNIST, CIFAR-10, and CIFAR-100, outperforming dense baselines with far fewer computations.

Problem

• Designing sparse, efficient artificial neural networks inspired by biology
• Reducing parameters while improving accuracy in vision benchmarks
• Incorporating biological connectivity patterns as structural bias

Innovation

• G2GNet architecture with sparse modular connectivity
• Dynamic sparse training mechanism pruning edges
• Hebbian-inspired rewiring rule based activation correlations