Differential Coding for Training-Free ANN-to-SNN Conversion

2025-03-01

AI Summary

To address the high energy consumption and long inference latency caused by spike redundancy in ANN-to-SNN conversion, this paper proposes a novel, training-free conversion paradigm that replaces conventional rate coding with differential encoding—encoding only activation changes—thereby substantially reducing spike density and temporal sequence length. Key innovations include: (i) the first ReLU-activation-distribution-driven iterative threshold optimization method; (ii) a cross-layer spike mapping mechanism; and (iii) a unified conversion framework compatible with both CNNs and Transformers. Evaluated on multiple mainstream architectures, the method achieves state-of-the-art accuracy–efficiency trade-offs: average spike count reduced by >40%, with concurrent reductions in inference latency and energy consumption, while strictly preserving the original ANN’s accuracy. By overcoming the inherent spatiotemporal redundancy of rate coding, this approach establishes a new pathway for efficient SNN deployment.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Diffusion Models for Vision

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Content-based information diffusion

Abstract

Spiking Neural Networks (SNNs) exhibit significant potential due to their low energy consumption. Converting Artificial Neural Networks (ANNs) to SNNs is an efficient way to achieve high-performance SNNs. However, many conversion methods are based on rate coding, which requires numerous spikes and longer time-steps compared to directly trained SNNs, leading to increased energy consumption and latency. This article introduces differential coding for ANN-to-SNN conversion, a novel coding scheme that reduces spike counts and energy consumption by transmitting changes in rate information rather than rates directly, and explores its application across various layers. Additionally, the threshold iteration method is proposed to optimize thresholds based on activation distribution when converting Rectified Linear Units (ReLUs) to spiking neurons. Experimental results on various Convolutional Neural Networks (CNNs) and Transformers demonstrate that the proposed differential coding significantly improves accuracy while reducing energy consumption, particularly when combined with the threshold iteration method, achieving state-of-the-art performance.

Problem

• Reduces spike counts and energy in ANN-to-SNN conversion
• Optimizes thresholds for ReLU to spiking neuron conversion
• Improves accuracy and energy efficiency in SNNs

Innovation

• Differential coding reduces spike counts and energy.
• Threshold iteration optimizes ReLU-to-spiking neuron conversion.
• Combined methods enhance accuracy and energy efficiency.