A concentration result for multilayer feedforward neural networks

2026-08-15

AI Summary

This study addresses the concentration of outputs in multi-layer feedforward neural networks under the large-width limit. Leveraging probabilistic limit theory and continuous function approximation, we establish an output concentration theorem applicable to networks of arbitrary depth, assuming weight distribution convergence and independent, identically distributed inputs. We demonstrate that as the number of neurons approaches infinity, the network output converges almost surely to a deterministic constant. This work overcomes the limitations of traditional theories restricted to shallow or specific architectures by revealing the asymptotic stability of deep network outputs. Consequently, these findings provide a novel mathematical foundation for deep learning theory, extending rigorous convergence guarantees to general deep architectures and offering critical insights into their behavior in the infinite-width regime.

Abstract

We consider for an arbitrary fixed $ρ$ and for each positive integer $n$ a multilayer feedforward artificial neural network with $ρ$ layers, $n$ neurons in the first layer (the input layer) and only one neuron, the output neuron, in the last layer. Very roughly formulated, the main result is that if the distribution of weights of connections from a layer to the next are, for all large $n$, approximated well by a fixed continuous (but otherwise arbitrary) curve which does not depend on $n$, and if the values of the $n$ input neurons are independently and identically distributed with a continuous probability density function, then there is a number $ψ$ such that for all $\varepsilon > 0$ the probability that the value of the output neuron is in $[ψ- \varepsilon, ψ+ \varepsilon]$ tends to 1 as $n$ tends to infinity.

Problem

• Multilayer feedforward neural networks
• Concentration result
• Asymptotic behavior
• Output neuron

Innovation

• Concentration result
• Multilayer feedforward neural networks
• Asymptotic convergence
• Weight distribution approximation
• Infinite-width limit