Continual Learning with Columnar Spiking Neural Networks

2025-06-20

AI Summary

To address catastrophic forgetting in continual learning, this paper proposes CoLaNET, a columnar-organized spiking neural network. Inspired by cortical microcolumn architecture, CoLaNET introduces, for the first time in SNN-based continual learning, a task-adaptive, non-shared microcolumn dynamic allocation mechanism. It integrates spike-timing-dependent plasticity (STDP) as a local learning rule with microcolumn-level activation/freeze strategies to explicitly balance stability and plasticity. Theoretical analysis demonstrates that the non-shared microcolumn structure achieves an optimal trade-off between adaptability and stability, and quantifies how key hyperparameters modulate this balance. Evaluated on a sequence of ten MNIST tasks, CoLaNET achieves a mean accuracy of 92% and exhibits only 4% forgetting on the first task—substantially outperforming state-of-the-art continual learning approaches.

Technology Category

Cognitive Modeling & Cognitive Systems: Neural Spike CodingMachine Learning: Life-Long and Continual LearningNatural Language Processing: Learning & Optimization for NLP

Application Category

Economics, Online Markets and Human Computation: Social networks and social learningSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingGraph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

This study investigates columnar-organized spiking neural networks (SNNs) for continual learning and catastrophic forgetting. Using CoLaNET (Columnar Layered Network), we show that microcolumns adapt most efficiently to new tasks when they lack shared structure with prior learning. We demonstrate how CoLaNET hyperparameters govern the trade-off between retaining old knowledge (stability) and acquiring new information (plasticity). Our optimal configuration learns ten sequential MNIST tasks effectively, maintaining 92% accuracy on each. It shows low forgetting, with only 4% performance degradation on the first task after training on nine subsequent tasks.

Problem

• Investigates columnar SNNs for continual learning and forgetting
• Explores CoLaNET hyperparameters balancing stability and plasticity
• Achieves high accuracy in sequential MNIST tasks

Innovation

• Columnar spiking networks for continual learning
• CoLaNET balances stability and plasticity
• Optimal config maintains high task accuracy