Threshold-Based Early Stopping of Accumulations in Neural Networks with Binary Activation

2026-08-06

AI Summary

Although binary neural networks require only the sign of outputs during inference, they conventionally perform full accumulation of all inputs, resulting in substantial redundant computation. This work introduces a practical early-termination strategy that exploits the observation that once partial sums in the accumulation process deviate sufficiently from zero, their final signs can be confidently predicted. The proposed method requires no retraining: by analyzing accumulation behavior on the training set, it dynamically sets thresholds to halt unnecessary additions early. Evaluated on VGG11 with CIFAR-10, the approach reduces up to 86.6% of accumulations in the deepest convolutional layer with only a 0.37% accuracy drop; when applied to the last three layers, overall arithmetic operations decrease by 25% at the cost of a 1.36% accuracy reduction.

Abstract

Binary neural networks are very attractive for constrained deployment, enabling small footprint and low-power inference. For binary activations, the dot products become sign-controlled additions or subtractions, but the number of operations is unchanged. Indeed, every neuron or output channel still accumulates all of its input, even though only the sign will be retained, which is often wasteful. As the accumulation progresses, the running partial sum frequently drifts so far from zero that its final sign becomes highly predictable long before the last term is reached; every contribution evaluated after that point changes the value of the sum but not the final output activation. This paper turns this observation into a post-training early-stopping mechanism. We characterize the behavior of the running accumulations on the training dataset and use this information to predict the final sign as soon as possible. No model parameter is retrained. We count the number of operations under an idealized ordering of weights. On VGG11 applied to the CIFAR-10 dataset, the method removes $86.6\%$ of the accumulation terms of the deepest convolution for a $0.37$-point accuracy drop, and $25\%$ of the full-network arithmetic when used on the three deepest convolutions simultaneously, for a $1.36$-point drop.

Problem

• binary neural networks
• early stopping
• accumulation
• binary activation
• computation efficiency

Innovation

• binary neural networks
• early stopping
• accumulation optimization
• sign prediction
• post-training acceleration