Technology Category
Machine Learning: Learning on the Edge & Model CompressionCognitive Modeling & Cognitive Systems: Neural Spike CodingSearch and Optimization: Learning to Search
Online Continual Learning via Spiking Neural Networks with Sleep Enhanced Latent Replay
🤖 AI Summary
Online continual learning for edge computing faces dual challenges: excessive memory overhead and classification bias toward new tasks. This paper proposes SESLR, an efficient online continual learning framework based on spiking neural networks (SNNs), which innovatively integrates implicit feature replay with a biologically inspired sleep-enhancement mechanism. Leveraging the binary spike nature of SNNs, SESLR stores replayed features using only one bit per feature; during a “sleep phase,” controlled noise injection strengthens consolidation of prior knowledge, effectively mitigating catastrophic forgetting and new-task bias. Experiments demonstrate that SESLR achieves nearly 30% higher average accuracy on Split CIFAR10 while reducing memory consumption to one-third of baseline methods; on CIFAR10-DVS, it improves accuracy by approximately 10% and cuts memory overhead by 32×. This work establishes a low-overhead, robust continual learning paradigm tailored for resource-constrained edge devices.
Technology Category
Application Category
📝 Abstract
Edge computing scenarios necessitate the development of hardware-efficient online continual learning algorithms to be adaptive to dynamic environment. However, existing algorithms always suffer from high memory overhead and bias towards recently trained tasks. To tackle these issues, this paper proposes a novel online continual learning approach termed as SESLR, which incorporates a sleep enhanced latent replay scheme with spiking neural networks (SNNs). SESLR leverages SNNs' binary spike characteristics to store replay features in single bits, significantly reducing memory overhead. Furthermore, inspired by biological sleep-wake cycles, SESLR introduces a noise-enhanced sleep phase where the model exclusively trains on replay samples with controlled noise injection, effectively mitigating classification bias towards new classes. Extensive experiments on both conventional (MNIST, CIFAR10) and neuromorphic (NMNIST, CIFAR10-DVS) datasets demonstrate SESLR's effectiveness. On Split CIFAR10, SESLR achieves nearly 30% improvement in average accuracy with only one-third of the memory consumption compared to baseline methods. On Split CIFAR10-DVS, it improves accuracy by approximately 10% while reducing memory overhead by a factor of 32. These results validate SESLR as a promising solution for online continual learning in resource-constrained edge computing scenarios.
Problem
Research questions and friction points this paper is trying to address.
Reduces memory overhead in online continual learning
Mitigates bias towards recent tasks in learning
Enhances edge computing adaptability via SNNs
Innovation
Methods, ideas, or system contributions that make the work stand out.
Uses Spiking Neural Networks for binary feature storage
Integrates sleep phase with noise for bias reduction
Achieves high accuracy with low memory consumption
🔎 Similar Papers
No similar papers found.
