Temporal Misalignment and Probabilistic Neurons

2025-02-20

AI Summary

To address performance instability in ANN-to-SNN conversion caused by time misalignment—i.e., temporal mismatches in spike emission—this paper proposes the Two-stage Probabilistic Spiking Neuron (TPP), the first framework to formally define and model this phenomenon. TPP enhances temporal robustness via a stochastic spike reordering mechanism, requiring no additional training overhead, and provides a theoretically grounded error bound analysis. The method is fully compatible with mainstream conversion frameworks. Extensive evaluations on CIFAR-10/100, CIFAR10-DVS, and ImageNet demonstrate its effectiveness: on ImageNet, TPP reduces the accuracy gap between ANN and SNN for ResNet-34 to just 1.2%, significantly outperforming existing state-of-the-art methods. Crucially, TPP preserves both biological interpretability and computational efficiency, offering a principled solution to temporal fragility in neuromorphic conversion.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Security and Privacy: Data transparency and provenance

Abstract

Spiking Neural Networks (SNNs) offer a more energy-efficient alternative to Artificial Neural Networks (ANNs) by mimicking biological neural principles, establishing them as a promising approach to mitigate the increasing energy demands of large-scale neural models. However, fully harnessing the capabilities of SNNs remains challenging due to their discrete signal processing and temporal dynamics. ANN-SNN conversion has emerged as a practical approach, enabling SNNs to achieve competitive performance on complex machine learning tasks. In this work, we identify a phenomenon in the ANN-SNN conversion framework, termed temporal misalignment, in which random spike rearrangement across SNN layers leads to performance improvements. Based on this observation, we introduce biologically plausible two-phase probabilistic (TPP) spiking neurons, further enhancing the conversion process. We demonstrate the advantages of our proposed method both theoretically and empirically through comprehensive experiments on CIFAR-10/100, CIFAR10-DVS, and ImageNet across a variety of architectures, achieving state-of-the-art results.

Problem

• Address temporal misalignment in ANN-SNN conversion
• Introduce probabilistic neurons to enhance SNN performance
• Achieve state-of-the-art results on benchmark datasets

Innovation

• Two-phase probabilistic spiking neurons
• ANN-SNN conversion enhancement
• Temporal misalignment utilization