🤖 AI Summary
This work addresses the inefficiency and weak theoretical guarantees of neural networks in approximating analytic functions and general $L^p$ functions. To overcome these limitations, the authors propose an efficient ReLU network architecture based on a three-dimensional design, which explicitly constructs sawtooth functions to achieve enhanced approximation capabilities. The proposed method significantly improves the exponential approximation rates for a broad class of analytic functions and, for the first time, establishes a high-order, non-asymptotic quantitative approximation theory for general $L^p$ functions. Notably, this approach achieves superior approximation performance while maintaining parameter efficiency and providing rigorous theoretical guarantees.
📝 Abstract
This work addresses two fundamental limitations in neural network approximation theory. We demonstrate that a three-dimensional network architecture enables a significantly more efficient representation of sawtooth functions, which serves as the cornerstone in the approximation of analytic and $L^p$ functions. First, we establish substantially improved exponential approximation rates for several important classes of analytic functions and offer a parameter-efficient network design. Second, for the first time, we derive a quantitative and non-asymptotic approximation of high orders for general $L^p$ functions. Our techniques advance the theoretical understanding of the neural network approximation in fundamental function spaces and offer a theoretically grounded pathway for designing more parameter-efficient networks.