🤖 AI Summary
This work investigates the approximation capabilities of single-hidden-layer neural networks with globally constrained coefficients—such as those bounded in ℓ¹ norm or exhibiting subexponential growth—when targeting non-analytic functions. Through a deterministic analysis that combines comparison arguments with Bernstein-type estimates, the study demonstrates that such networks remain “rigid” under Gevrey-class activation functions: their approximation error is fundamentally governed by the best polynomial approximation rate, with only exponentially small residual terms. The findings establish that, even when employing non-analytic activations, shallow networks with coefficient constraints cannot surpass classical polynomial approximation rates, thereby revealing an intrinsic limitation in their representational power.
📝 Abstract
We study approximation limits of single-hidden-layer neural networks with analytic activation functions under global coefficient constraints. Under uniform $\ell^1$ bounds, or more generally sub-exponential growth of the coefficients, we show that such networks generate model classes with strong quantitative regularity, leading to uniform analyticity of the realized functions. As a consequence, up to an exponentially small residual term, the error of best network approximation on generic target functions is bounded from below by the error of best polynomial approximation. In particular, networks with analytic activation functions with controlled coefficients cannot outperform classical polynomial approximation rates on non-analytic targets. The underlying rigidity phenomenon extends to smoother, non-analytic activations satisfying Gevrey-type regularity assumptions, yielding sub-exponential variants of the approximation barrier. The analysis is entirely deterministic and relies on a comparison argument combined with classical Bernstein-type estimates; extensions to higher dimensions are also discussed.