All in one timestep: Enhancing Sparsity and Energy efficiency in Multi-level Spiking Neural Networks

2025-10-28

AI Summary

Spiking Neural Networks (SNNs) suffer from severe information loss due to their inherent binary spike mechanism, limiting both accuracy and energy efficiency. To address this, we propose a multi-level spiking neuron model that employs fine-grained spike coding to reduce quantization error, and introduce Sparse-ResNet—a novel architecture that mitigates spike avalanche via sparse residual connections, integrated with event-driven computation and ultra-low-bit weight activation quantization. Notably, our approach achieves single-timestep inference on neuromorphic datasets for the first time, reducing inference latency by 10×. Evaluated on mainstream image classification and neuromorphic benchmarks, it attains accuracy comparable to full-precision artificial neural networks (ANNs), improves energy efficiency by 2–3×, and reduces network activation by over 20%. This work advances brain-inspired computing toward low-latency, high-accuracy, and high-energy-efficiency operation.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Spiking Neural Networks (SNNs) are one of the most promising bio-inspired neural networks models and have drawn increasing attention in recent years. The event-driven communication mechanism of SNNs allows for sparse and theoretically low-power operations on dedicated neuromorphic hardware. However, the binary nature of instantaneous spikes also leads to considerable information loss in SNNs, resulting in accuracy degradation. To address this issue, we propose a multi-level spiking neuron model able to provide both low-quantization error and minimal inference latency while approaching the performance of full precision Artificial Neural Networks (ANNs). Experimental results with popular network architectures and datasets, show that multi-level spiking neurons provide better information compression, allowing therefore a reduction in latency without performance loss. When compared to binary SNNs on image classification scenarios, multi-level SNNs indeed allow reducing by 2 to 3 times the energy consumption depending on the number of quantization intervals. On neuromorphic data, our approach allows us to drastically reduce the inference latency to 1 timestep, which corresponds to a compression factor of 10 compared to previously published results. At the architectural level, we propose a new residual architecture that we call Sparse-ResNet. Through a careful analysis of the spikes propagation in residual connections we highlight a spike avalanche effect, that affects most spiking residual architectures. Using our Sparse-ResNet architecture, we can provide state-of-the-art accuracy results in image classification while reducing by more than 20% the network activity compared to the previous spiking ResNets.

Problem

• Reducing information loss in SNNs while maintaining accuracy
• Achieving low energy consumption and minimal inference latency
• Preventing spike avalanches in spiking residual architectures

Innovation

• Multi-level spiking neurons reduce quantization error
• Sparse-ResNet architecture minimizes network activity spikes
• One-timestep inference drastically cuts energy consumption