backpropagation through time

This study systematically investigates three core challenges in spiking neural network (SNN) training: (1) gradient estimation difficulty arising from the non-differentiable spiking mechanism; (2) high computational/memory overhead and biological implausibility of backpropagation through time (BPTT); and (3) the trade-off between biological plausibility and task performance inherent in local learning rules. To address these, we conduct a unified empirical evaluation of learning methods with varying degrees of locality—including STDP and e-prop—and introduce an explicit recurrent weight design. We uncover shared training dynamics across local methods for the first time. Experiments demonstrate that explicit recurrence significantly enhances robustness, improving adversarial accuracy on CIFAR-10 by 12.3% on average. Moreover, we pioneer the assessment of local learning rules under both FGSM (white-box gradient-based) and NES (black-box) attacks: under NES, local methods retain >68% accuracy—surpassing BPTT—thereby establishing their superior generalization to biologically plausible, resource-efficient, and robust SNN training.

Spiking Neural Networks (SNNs) pose challenges for standard gradient-based optimization due to the discontinuous, event-driven nature of spiking dynamics; conventional surrogate gradient methods introduce approximation errors that compromise training fidelity. Method: We propose an endpoint-blanking continuous spiking dynamics model, wherein spike initiation and termination are smoothly modulated at trial boundaries—enabling exact, differentiable spike generation and rigorous backpropagation through time. Contribution/Results: This is the first method to achieve fully differentiable, pointwise-exact gradient descent in SNNs—eliminating surrogate approximations entirely. It supports gradient-driven dynamic insertion and deletion of spikes, and generalizes to deep and recurrent architectures—even those initialized in silence. Experiments demonstrate stable convergence across multiple tasks, with substantial improvements in training accuracy and robustness, thereby overcoming fundamental theoretical limitations inherent in surrogate gradient approaches.

Spiking neural networks (SNNs) suffer from severe overfitting, limiting their generalization capability. To address this, we propose Temporal Reversal Regularization (TRR), the first method to explicitly model temporal reversibility as a regularization prior for SNNs. TRR enforces consistency between spike-rate distributions of original and temporally reversed inputs/feature sequences, while incorporating lightweight Hadamard-based feature mixing to construct spatiotemporal-invariant representations. We theoretically derive a tightened upper bound on the generalization error. Empirically, TRR significantly improves generalization and adversarial robustness across static image classification, neuromorphic event-stream recognition, and 3D point cloud classification. Notably, it achieves substantial accuracy gains in low-latency neuromorphic object recognition. By enhancing both performance and efficiency, TRR advances brain-inspired co-design of algorithms and neuromorphic hardware.

Standard backpropagation through time (BPTT) and its truncated or higher-order approximations suffer from significant gradient bias and unstable convergence in RNNs due to long-range dependencies. To address this, we propose an exact backward propagation method grounded in discrete forward sensitivity equations (DFSE). This is the first work to integrate DFSE into RNN training, enabling unbiased, full-sequence gradient computation while natively supporting time-varying parameters and multi-cycle coupled architectures. By performing precise Jacobian chain propagation, our method eliminates truncation errors and avoids cumulative bias from higher-order approximations. Experiments on long-sequence tasks demonstrate substantial improvements in gradient accuracy and training stability. Our approach establishes a new paradigm for modeling strong long-term dependencies in recurrent systems.

This work investigates learning stability and generalization of spiking neural networks (SNNs) under nonnegative weight constraints. We propose a novel architecture integrating affine encoders-decoders with nonnegative-weight spiking neurons, and analyze it via covering number theory and Barron function approximation theory. Our analysis establishes, for the first time, a depth-independent generalization bound for such SNNs; theoretically achieves rate-optimal approximation of ReLU networks; and overcomes the fundamental bottleneck wherein conventional SNN generalization degrades with depth. The design ensures parameter continuity and gradient-descent training stability. Experiments on standard benchmarks demonstrate competitive accuracy and—crucially—a near-constant generalization error across increasing depths, empirically validating our theoretical predictions. The core contribution is the first provably generalizable, trainably stable, and depth-robust framework for nonnegative-weight SNNs.

This work addresses the long-standing challenges in spiking neural network (SNN) training research—namely, the lack of a systematic taxonomy and unified evaluation protocols, which have led to poor reproducibility and fragmented progress. To this end, we propose the first fine-grained, systematic classification framework for SNN training algorithms and introduce NeuroTrain, an open-source benchmark built upon snnTorch. NeuroTrain enables, for the first time, modular integration and fair comparison across diverse algorithmic paradigms, including surrogate gradient backpropagation, local and three-factor learning rules, biologically plausible plasticity mechanisms, ANN-to-SNN conversion methods, and unconventional optimization strategies. The framework standardizes the implementation of representative algorithms and supports consistent evaluation across datasets, architectures, and training configurations, thereby significantly enhancing the reproducibility and systematic investigation of SNN training methodologies.

This work addresses the challenge of deploying conventional recurrent spiking neural networks (SNNs) on resource-constrained edge devices, where large parameter counts and slow inference hinder the balance between efficiency and accuracy. To overcome this limitation, the authors propose a novel convolutional recurrent SNN architecture that, for the first time, integrates convolutional recurrent connections with a learnable axonal delay mechanism, accompanied by a tailored training methodology. Evaluated on audio classification tasks, the proposed model achieves a dramatic reduction in model size—cutting recurrent parameters by approximately 99%—while delivering a 52× speedup in inference latency, all without compromising classification accuracy. This advancement establishes a new paradigm for efficient edge intelligence with spiking neural networks.

This work addresses the limited representational capacity of spiking neural networks (SNNs) caused by the inherent inconsistency in temporal spike dynamics. To overcome this, the authors propose a dual-consistency optimization framework: first, a hardware-friendly bitwise AND operation is employed to efficiently decouple a stable spike skeleton from multi-timestep spike trains, enforcing unstable spikes to converge toward this consistent structure and thereby enhancing temporal coherence; second, magnitude-aware spike noise is introduced to enrich representational diversity and improve generalization. Notably, this is the first approach to leverage bitwise AND operations for both spike skeleton extraction and noise injection. The method achieves significant performance gains under ultra-low latency settings across diverse architectures and datasets, with accuracy improvements up to 8.33%, effectively unlocking the energy efficiency and speed potential of SNNs.

This work addresses the fundamental limitation of conventional neural networks, which rely on repeated data access and epoch-based training, thereby losing long-term consistency in irreversible data streams and degenerating into reactive filters devoid of temporal memory. To overcome this, the paper introduces Streaming Neural Networks (StNN), equipped with a stream-native execution algorithm (SNA) and streaming neurons that maintain persistent temporal states, enabling continuous, epoch-free learning. The study establishes the first minimal computational foundation for neural processing of irreversible streaming data, theoretically proving that the network’s state dynamics exhibit boundedness and contractivity. Through phase-space analysis and temporal modeling, the authors further demonstrate the stability and effectiveness of StNN in long-horizon stream processing tasks.