🤖 AI Summary
This work addresses the sensitivity of conventional neurons—whose linear weighted summation is vulnerable to noise—and its detrimental impact on model robustness. To overcome this limitation, the study introduces, for the first time at the neuronal level, a learnable nonlinear aggregation mechanism by proposing F-Mean neurons and Gaussian-support neurons. A differentiable hybrid architecture is constructed to enable end-to-end training that adaptively interpolates between these two neuron types. The approach demonstrates that the choice of aggregation function constitutes a critical design dimension for enhancing noise robustness. Experiments on CIFAR-10 and its noisy variants show that the triple-hybrid model substantially improves robustness (achieving a score of 0.991 versus the baseline 0.890) while also yielding modest gains on clean data. Learned parameters consistently converge to sublinear aggregation (p ≈ 0.43–0.50) and high utilization of input novelty (α ≈ 0.69–0.79).
📝 Abstract
Weighted summation has remained the default input aggregation mechanism in artificial neurons since the earliest neural network models. While computationally efficient, this design implicitly behaves like a mean-based estimator and is therefore sensitive to noisy or extreme inputs. This paper investigates whether replacing fixed linear aggregation with learnable nonlinear alternatives can improve neural network robustness without sacrificing trainability. Two differentiable aggregation mechanisms are introduced: an F-Mean neuron based on a learnable power-weighted aggregation rule, and a Gaussian Support neuron based on distance-aware affinity weighting. To preserve the optimisation stability of standard neurons, hybrid neurons are proposed that interpolate between linear and nonlinear aggregation through a learnable blending parameter. Evaluated in multilayer perceptrons and convolutional neural networks on CIFAR-10 and a noisy CIFAR-10 variant with additive Gaussian corruption, hybrid neurons consistently improve robustness under noise while F-Mean hybrids also yield modest gains on clean data. The three-way hybrid achieves robustness scores of up to 0.991 compared to 0.890 for the standard baseline, and learned parameters converge consistently to sub-linear aggregation (p $\approx$ 0.43--0.50) and high novelty utilisation ($α$ $\approx$ 0.69--0.79). These findings suggest that neuron-level aggregation is a meaningful and underexplored design dimension for building more noise-tolerant neural networks.