Problem
Research questions and friction points this paper is trying to address.
spectral Barron functions
deep ReLU neural networks
approximation bounds
minimax rates
curse of dimensionality
Minimax rates for learning spectral Barron functions by deep ReLU neural networks
🤖 AI Summary
This study addresses the open problem of characterizing the approximation capabilities of deep ReLU networks for spectral Barron functions and establishing minimax optimal rates in statistical learning. By integrating approximation theory with statistical learning theory, this work derives novel approximation bounds for deep networks over this function class and formulates sample-size-dependent learning rates. The primary contribution lies in closing a theoretical gap by providing parameter-dependent approximation rates and a sample-dependent learning rate of $n^{-\frac{d+2s}{2d+2s}}$ for spectral Barron functions. Furthermore, this convergence rate is rigorously proven to be minimax optimal. These findings offer fundamental insights into the expressive power and generalization performance of deep neural networks within nonparametric regression settings governed by spectral Barron spaces.
📝 Abstract
We study how well deep neural networks approximate and learn spectral Barron functions. Recent studies have shown that these function classes can be efficiently approximated by shallow neural networks without suffering from the curse of dimensionality. We complement these results by providing new approximation bounds for deep networks with ReLU activation and establishing the minimax rates for learning these function classes. Specifically, we show that $d$-dimensional spectral Barron functions with smoothness index $s>0$ can be approximated by deep ReLU neural networks with approximation rate $\widetilde{\mathcal{O}} (S^{-\frac{1}{2}-\frac{s}{d}})$, where $S$ denotes the number of nonzero parameters in the network. Using this approximation result, we further show that deep ReLU neural networks can learn spectral Barron functions in a fast rate $n^{-\frac{d+2s}{2d+2s}}$ with $n$ training samples. Finally, we prove that this convergence rate is minimax optimal up to logarithmic factors.
🔎 Similar Papers
No similar papers found.
