spiking knowledge distillation

To address the low training efficiency of spiking neural networks (SNNs) and the difficulty of effectively transferring knowledge from artificial neural networks (ANNs) into the rate-coding domain, this paper proposes an ANN-guided, end-to-end differentiable distillation framework. The method replaces SNN components with corresponding ANN modules in a block-wise manner, embedding them directly into the SNN forward pass—thereby preserving intrinsic spiking dynamics while enabling progressive alignment of rate-coded feature representations. Crucially, it is the first to seamlessly integrate rate-coded backpropagation into such hybrid architectures, ensuring both gradient validity and structural consistency. Evaluated on multiple benchmark datasets, the approach significantly accelerates training convergence and improves generalization performance, consistently outperforming state-of-the-art ANN-to-SNN distillation methods.

The fundamental discrepancy between the continuous output distribution of artificial neural networks (ANNs) and the sparse, discrete spike-based outputs of spiking neural networks (SNNs) severely limits knowledge distillation performance. To address this, we propose a novel distillation paradigm tailored for SNNs. Our method comprises two core components: (1) saliency-scaled activation map distillation, which enables semantic-aware alignment of intermediate feature representations by weighting activations according to input saliency; and (2) noise-smoothed logits distillation, which injects Gaussian noise into SNN logits to mitigate gradient instability arising from output sparsity and discreteness. Integrated within an ANN-to-SNN conversion framework, our approach consistently improves both accuracy and convergence speed across CIFAR-10, CIFAR-100, and ImageNet. Compared to state-of-the-art distillation methods, it achieves average accuracy gains of 2.1–4.7 percentage points. The implementation is publicly available.

Large performance gaps persist between spiking neural networks (SNNs) and artificial neural networks (ANNs), exacerbated by existing knowledge distillation (KD) methods that neglect the intrinsic spatiotemporal dynamics of SNNs. To address this, we propose a temporally decoupled logit-level KD framework. Our method explicitly decomposes the distillation process across timesteps at the logit level and introduces class-probability entropy regularization to stabilize optimization and enhance temporal representation robustness. Unlike conventional approaches that aggregate outputs over time, our framework achieves fine-grained timestep-wise logit alignment—uniquely unlocking the full temporal expressive capacity of SNNs. Evaluated on multiple benchmark datasets, the proposed method consistently outperforms state-of-the-art logit-, feature-, and hybrid-based KD approaches, significantly improving classification accuracy while preserving the SNN’s inherent energy efficiency.

To address the challenge of simultaneously achieving biological plausibility, hardware efficiency, and competitive performance in spiking neural networks (SNNs) for general supervised classification, this paper proposes a columnar hierarchical SNN architecture tailored for classification. It employs intra-class-difference-driven columnar organization—each column represents a discriminative subcategory—and adopts an all-spiking signal flow with functionally specialized neurons. A biologically grounded learning mechanism is introduced, integrating local anti-Hebbian plasticity with dopamine neuromodulation to replace backpropagation entirely. The method unifies model-driven reinforcement learning with a state-proximity evaluation framework, enabling end-to-end training directly in the spike domain. Experiments demonstrate that the architecture achieves high accuracy and strong generalization across multiple benchmark classification tasks, while exhibiting exceptional compatibility with low-power neuromorphic hardware. This work establishes a novel paradigm for practical, deployable SNNs.

This study systematically investigates three core challenges in spiking neural network (SNN) training: (1) gradient estimation difficulty arising from the non-differentiable spiking mechanism; (2) high computational/memory overhead and biological implausibility of backpropagation through time (BPTT); and (3) the trade-off between biological plausibility and task performance inherent in local learning rules. To address these, we conduct a unified empirical evaluation of learning methods with varying degrees of locality—including STDP and e-prop—and introduce an explicit recurrent weight design. We uncover shared training dynamics across local methods for the first time. Experiments demonstrate that explicit recurrence significantly enhances robustness, improving adversarial accuracy on CIFAR-10 by 12.3% on average. Moreover, we pioneer the assessment of local learning rules under both FGSM (white-box gradient-based) and NES (black-box) attacks: under NES, local methods retain >68% accuracy—surpassing BPTT—thereby establishing their superior generalization to biologically plausible, resource-efficient, and robust SNN training.

This work addresses the long-standing challenges in spiking neural network (SNN) training research—namely, the lack of a systematic taxonomy and unified evaluation protocols, which have led to poor reproducibility and fragmented progress. To this end, we propose the first fine-grained, systematic classification framework for SNN training algorithms and introduce NeuroTrain, an open-source benchmark built upon snnTorch. NeuroTrain enables, for the first time, modular integration and fair comparison across diverse algorithmic paradigms, including surrogate gradient backpropagation, local and three-factor learning rules, biologically plausible plasticity mechanisms, ANN-to-SNN conversion methods, and unconventional optimization strategies. The framework standardizes the implementation of representative algorithms and supports consistent evaluation across datasets, architectures, and training configurations, thereby significantly enhancing the reproducibility and systematic investigation of SNN training methodologies.

Traditional artificial neural networks (ANNs) suffer from low energy efficiency and slow dynamic response, limiting their suitability for edge and neuromorphic computing. Method: This study systematically investigates modeling and training mechanisms of leaky integrate-and-fire (LIF) spiking neural networks (SNNs) for brain-inspired computing. We propose a multi-strategy comparative framework integrating surrogate gradient descent, ANN-to-SNN conversion, and spike-timing-dependent plasticity (STDP), evaluated across accuracy, energy consumption (mJ/inference), latency (ms), and convergence epochs. Results: Surrogate gradient training achieves near-ANN accuracy (within 1–2% degradation), converges by epoch 20, and attains inference latency as low as 10 ms; STDP yields ultra-low energy consumption (5 mJ/inference), enabling unsupervised learning and ultra-low-power edge deployment. This work quantitatively characterizes, for the first time, the fundamental trade-offs among accuracy, energy efficiency, and latency across SNN training paradigms—providing a deployable optimization roadmap for robotic perception, neuromorphic vision, and edge AI.

This work addresses a key limitation in existing knowledge distillation methods for spiking neural networks (SNNs), which uniformly align teacher and student outputs across all time steps, disregarding variations in prediction quality over time. To overcome this, the authors propose Selective Alignment Knowledge Distillation (SeAl-KD), which introduces, for the first time, an error-aware and confidence-driven mechanism to selectively align only those time steps exhibiting low confidence or high prediction error. During these critical intervals, competitive logits are dynamically corrected, while cross-time-step similarity-based reweighting preserves informative temporal dynamics. By departing from the conventional assumption of uniform distillation, SeAl-KD achieves substantial performance gains over prior approaches on both static image and neuromorphic event datasets, effectively narrowing the accuracy gap between SNNs and artificial neural networks (ANNs).

Spiking Neural Networks (SNNs) face significant challenges in scaling self-supervised learning to large-scale unlabeled data, primarily because spike discreteness disrupts cross-view gradient consistency, hindering optimization of contrastive and consistency objectives. Method: We propose a dual-path neuron architecture that jointly integrates a differentiable surrogate branch—enabling gradient propagation during training—and a genuine spiking branch—preserving full spike dynamics during inference. Coupled with cross-view and temporal alignment losses, this design enhances inter-sample representation consistency within both convolutional and Transformer-based SNNs. Contribution/Results: This work achieves the first full self-supervised pretraining of SNNs at ImageNet scale. Our Spikformer-16-512 model attains 70.1% top-1 accuracy on ImageNet-1K, demonstrating the feasibility of high-capacity SNNs for unsupervised learning at modern scales.

Local field potential (LFP) signals suffer from low modeling fidelity and poor cross-session generalization due to their inherent population-level aggregation, limiting their utility in downstream applications such as motor decoding. To address this, we propose the first spike-to-LFP cross-modal, representation-level knowledge distillation framework. Our method leverages a multi-session pretrained spike Transformer to extract high-fidelity neural representations, integrates session-specific neural tokenization with latent-space alignment, and enables label-free, high-fidelity knowledge transfer. We employ masked autoencoding pretraining, adaptive tokenization, and joint unsupervised-supervised optimization. Experiments demonstrate that the distilled LFP model significantly outperforms both single- and multi-session baselines across fully unsupervised and supervised tasks (p < 0.001), achieves robust cross-session generalization, and improves motor behavior decoding accuracy by 12.7%. This work establishes a principled, scalable paradigm for leveraging high-resolution spiking activity to enhance low-resolution LFP modeling.