Rethinking Spiking Neural Networks from an Ensemble Learning Perspective

2025-02-20

AI Summary

Spiking Neural Networks (SNNs) exhibit high energy efficiency but suffer from limited performance, primarily due to large inter-timestep variations in neuronal membrane potentials—causing instability in subnetwork outputs. This work pioneers an ensemble-learning perspective, modeling SNNs as temporally unfolded subnetworks sharing weights. We propose a membrane potential smoothing mechanism and an adjacent-subnetwork guidance strategy to mitigate gradient vanishing and enhance temporal consistency. The method requires no architectural modification and supports end-to-end training. On CIFAR10-DVS, it achieves 83.20% accuracy using only four timesteps. Moreover, consistent performance gains are demonstrated across diverse modalities—including speech, image, and point cloud recognition—validating its generality and effectiveness.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Learning with Manifolds
• Computer Vision: Learning & Optimization for CV

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Social networks and social learning

Abstract

Spiking neural networks (SNNs) exhibit superior energy efficiency but suffer from limited performance. In this paper, we consider SNNs as ensembles of temporal subnetworks that share architectures and weights, and highlight a crucial issue that affects their performance: excessive differences in initial states (neuronal membrane potentials) across timesteps lead to unstable subnetwork outputs, resulting in degraded performance. To mitigate this, we promote the consistency of the initial membrane potential distribution and output through membrane potential smoothing and temporally adjacent subnetwork guidance, respectively, to improve overall stability and performance. Moreover, membrane potential smoothing facilitates forward propagation of information and backward propagation of gradients, mitigating the notorious temporal gradient vanishing problem. Our method requires only minimal modification of the spiking neurons without adapting the network structure, making our method generalizable and showing consistent performance gains in 1D speech, 2D object, and 3D point cloud recognition tasks. In particular, on the challenging CIFAR10-DVS dataset, we achieved 83.20% accuracy with only four timesteps. This provides valuable insights into unleashing the potential of SNNs.

Problem

• Improve SNN performance via ensemble learning
• Stabilize initial membrane potential distribution
• Mitigate temporal gradient vanishing issue

Innovation

• Membrane potential smoothing technique
• Temporally adjacent subnetwork guidance
• Minimal modification of spiking neurons