Arbitrary-Accuracy Neural Approximation with Optimal Neuron Count and Near-Optimal Bit Complexity

2026-09-28

AI Summary

This study addresses the problem of minimizing neuron count and optimizing encoding complexity for arbitrary-precision approximation of multivariate Hölder continuous functions. Based on standard feedforward networks and multilayer perceptron architectures, the proposed method employs an explicitly defined fixed activation function, integrating grid addressing techniques with an integer encoding mechanism for quantized values to construct a closed-form network. The authors rigorously prove that the lower bound on the total number of neurons is d+1 and present a simplified construction requiring only two neurons beyond this bound. Furthermore, the approach achieves a bit complexity of O(ε^{-d/α}log(1/ε)), matching the metric entropy lower bound. These results validate the feasibility of attaining near-optimal bit complexity for arbitrary-precision approximation using a minimal number of neurons.

Abstract

We study the minimum number of hidden neurons required for arbitrary-accuracy approximation of multivariate H\"older-continuous functions on $[0,1]^d$ and the associated encoding complexity. For $d\geq 2$, we construct a fixed, explicitly defined activation function for which a closed-form network with two hidden layers of widths $d$ and $1$ achieves arbitrary accuracy in the uniform norm. We prove that $d+1$ is the exact minimum total number of hidden neurons among standard feedforward networks with locally integrable activations and affine outputs. We further give a simpler construction using a single elementary activation that combines the floor and exponential functions. This construction requires three hidden layers of widths $d$, $1$, and $2$, only two neurons above the minimum. If a skip connection is allowed, widths $d$, $1$, and $1$ suffice. These constructions use explicit grid addressing and integer encoding of quantized function values. For a bounded $\alpha$-H\"older class, they require $O(\varepsilon^{-d/\alpha}\log(1/\varepsilon))$ bits, matching the metric-entropy lower bound up to a logarithmic factor.

Problem

• neural network approximation
• minimum neuron count
• Hölder-continuous functions
• bit complexity
• metric entropy

Innovation

• arbitrary-accuracy approximation
• optimal neuron count
• bit complexity
• Hölder-continuous functions
• activation function construction