Effective pruning of task-trained recurrent neural networks using noisy fluctuations and connection rescaling

2026-08-05

AI Summary

Existing pruning methods struggle to simultaneously preserve task performance and biological plausibility, particularly in functional recurrent neural networks. This work proposes and empirically validates noise-prune, a novel local pruning approach based on synaptic noise fluctuations: it retains critical connections through local sampling and rescales their weights to maintain average synaptic strength. The study demonstrates that both the connection sampling strategy and the weight rescaling mechanism are essential for performance, and revises the theoretically predicted optimal rescaling magnitude. Noise-prune significantly outperforms magnitude-only pruning strategies and matches or even exceeds the performance of non-local methods that rely on second-order information, while remaining computationally efficient and biologically plausible.

Abstract

The pruning of network connections is key to brain function but, despite its importance, there exist few biologically-plausible pruning rules with demonstrated good performance. In this work we evaluate noise-prune, a recently introduced unsupervised local pruning rule for recurrent networks that uses noisy fluctuations to determine the importance of connections. Noise-prune has previously only been empirically tested on random networks without a specific computational function. We show that noise-prune preserves task-performance in task-trained recurrent neural networks, greatly outperforming a strategy that only uses the magnitude of connections and performing on par with or exceeding a non-local strategy that uses second-order information. Rather than deterministically removing connections that fall below a certain threshold importance, noise-prune samples connections to preserve based on their importance and strengthens retained connections to preserve average synaptic strength. We show that this sampling and rescaling is essential to good performance, but that the optimal empirical degree of rescaling is lower than that predicted by the original theoretical argument. Our work thus validates noise-prune as a biologically-plausible pruning rule for functional recurrent network architectures and characterizes its optimal parameter settings.

Problem

• pruning
• recurrent neural networks
• biologically-plausible
• task-performance
• synaptic strength

Innovation

• noise-prune
• recurrent neural networks
• biologically-plausible pruning
• connection rescaling
• unsupervised pruning