Predictive Suppression Layers for Communication-Efficient Spiking Neural Networks

2026-09-18

AI Summary

为解决尖峰神经网络中冗余信息传输问题，提出了一种预测编码框架，通过动态门控仅传递非冗余的'惊喜'活动，实现了通信节省并提高了任务准确性。

Abstract

Feedforward Spiking Neural Networks (SNNs) typically propagate every generated spike indiscriminately, disregarding whether the information is redundant from an information-theoretic perspective. This lack of selectivity induces high redundancy in inter-layer communication, creating an expensive overhead, e.g., in scenarios involving many-core neuromorphic hardware or communication-dominated Internet-of-Things (IoT) where features are transmitted wirelessly. To address this challenge, we trade localized processing for leaner network channels by introducing a minimal predictive coding framework for SNNs. We propose two layer variants sharing a predictor block: error units, which transmit signed spiking residuals, and predictive suppression, which uses residual magnitude to dynamically gate and forward only unpredictable, "surprising" activity. Evaluated on the N-MNIST and Spiking Heidelberg Digits (SHD) datasets using diagnostic metrics that decouple local processing from cross-layer communication, our new predictive coding layers achieve significant communication savings. Numerical results reveal a three-fold reduction in communicated activity, while increasing the task accuracy for both datasets. The latter finding is notable, and suggests that predictive coding layers not only minimize communication overhead, but also produce output feature vectors with a higher representation power.

Problem

• SNNs
• redundancy
• communication overhead
• neuromorphic hardware
• IoT

Innovation

• predictive coding
• spiking neural networks
• communication efficiency
• redundancy reduction