Spiking Layer-Adaptive Magnitude-based Pruning

2026-03-16

AI Summary

This work addresses the limitations of conventional magnitude-based pruning methods in spiking neural networks (SNNs), which neglect temporal accumulation effects, non-uniform contributions across time steps, and membrane potential stability, leading to significant performance degradation. To overcome these issues, the authors propose SLAMP, a novel framework that extends layer-adaptive pruning into the temporal domain of SNNs for the first time. By formulating a distortion-constrained optimization problem that incorporates time-aware sensitivity, SLAMP jointly allocates sparsity across both network layers and individual time steps. The method introduces a time-aware layer importance metric and naturally reduces to classical pruning strategies in the single-time-step limit, ensuring both theoretical rigor and practical applicability. Experiments on CIFAR10, CIFAR100, and CIFAR10-DVS demonstrate that SLAMP substantially reduces synaptic connections and spike operations while preserving high accuracy, thereby enhancing the deployment efficiency of SNNs.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Machine Learning: Learning on the Edge & Model Compression

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Spiking Neural Networks (SNNs) provide energy-efficient computation but their deployment is constrained by dense connectivity and high spiking operation costs. Existing magnitude-based pruning strategies, when naively applied to SNNs, fail to account for temporal accumulation, non-uniform timestep contributions, and membrane stability, often leading to severe performance degradation. This paper proposes Spiking Layer-Adaptive Magnitude-based Pruning (SLAMP), a theory-guided pruning framework that generalizes layer-adaptive magnitude pruning to temporal SNNs by explicitly controlling worst-case output distortion across layers and timesteps. SLAMP formulates sparsity allocation as a temporal distortion-constrained optimization problem, yielding time-aware layer importance scores that reduce to conventional layer-adaptive pruning in single-timestep limit. An efficient two-stage procedure is derived, combining temporal score estimation, global sparsity allocation, and magnitude pruning with retraining for stability recovery. Experiments on CIFAR10, CIFAR100, and the event-based CIFAR10-DVS datasets demonstrate that SLAMP achieves substantial connectivity and spiking operation reductions while preserving accuracy, enabling efficient and deployable SNN inference.

Problem

• Spiking Neural Networks
• magnitude-based pruning
• temporal accumulation
• membrane stability
• spiking operation costs

Innovation

• Spiking Neural Networks
• Magnitude-based Pruning
• Layer-adaptive
• Temporal Distortion
• Sparsity Allocation