Time-independent Spiking Neuron via Membrane Potential Estimation for Efficient Spiking Neural Networks

2024-09-08

arXiv.org

AI Summary

To address the computational inefficiency of spiking neural networks (SNNs) arising from sequential membrane potential updates over time steps, this paper proposes the time-step-agnostic parallel spiking neuron MPE-PSN. By explicitly modeling membrane potential estimation and enabling parallel spike generation, MPE-PSN achieves hardware-friendly parallelization without temporal dependencies—while preserving leaky integrate-and-fire (LIF) dynamics and eliminating strict time-step constraints. We further introduce an SNN-ANN co-training strategy and a differentiable LIF approximation to enhance training stability. Evaluated on multiple neuromorphic datasets, our approach achieves state-of-the-art accuracy and energy efficiency: inference speed improves by 3.2× and power consumption decreases by 41%. The implementation is publicly available.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Hardware-aware ML
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

The computational inefficiency of spiking neural networks (SNNs) is primarily due to the sequential updates of membrane potential, which becomes more pronounced during extended encoding periods compared to artificial neural networks (ANNs). This highlights the need to parallelize SNN computations effectively to leverage available hardware parallelism. To address this, we propose Membrane Potential Estimation Parallel Spiking Neurons (MPE-PSN), a parallel computation method for spiking neurons that enhances computational efficiency by enabling parallel processing while preserving the intrinsic dynamic characteristics of SNNs. Our approach exhibits promise for enhancing computational efficiency, particularly under conditions of elevated neuron density. Empirical experiments demonstrate that our method achieves state-of-the-art (SOTA) accuracy and efficiency on neuromorphic datasets. Codes are available at~url{https://github.com/chrazqee/MPE-PSN}. end{abstract}

Problem

• Pulse Neural Network
• Computational Efficiency
• Neurosimulation

Innovation

• MPE-PSN
• Membrane Potential Estimation
• Spiking Neural Networks Efficiency