Problem
Research questions and friction points this paper is trying to address.
ReLU neural networks
Sobolev norm
approximation error
path norm constraint
Error bounds in Sobolev norms for approximations with norm constrained ReLU neural networks
๐ค AI Summary
็ ็ฉถไบๅจSobolev่ๆฐไธ๏ผ้่ฟ้ๅถๆ้่ทฏๅพ่ๆฐ็ReLU็ฅ็ป็ฝ็ปๅฏนๅ
ๆปๅฝๆฐ็้ผ่ฟ่ฏฏๅทฎ็ใ
๐ Abstract
Recent studies have shown that smooth functions can be well approximated by ReLU neural networks with path norm constraint on the weights. We extend these results from uniform approximation to approximation in Sobolev norm. Specifically, we analyze how well Sobolev functions in $W^{n,p}$ can be approximated by neural networks with width $W$, depth $L$ and path norm bounded by $K$, when the approximation error is measured in the $W^{1,p}$-norm. For shallow networks with depth $L=1$, we derive the approximation error bound $\mathcal{O}(\max\{W^{-(n-1)/d}, K^{-(n-1)/(s-n)}\})$, when the smoothness index satisfies $n<s=(d+3)/2$ and the input is $d$-dimensional. For deep networks, we remove the restriction on the smoothness by showing that the approximation bound $\mathcal{O}(K^{-(n-1)/(d+d/p+1)})$ holds if the width $W$ and depth $L$ are sufficiently large.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Sobolev norm
ReLU neural networks
approximation error bound
path norm constraint
๐ Similar Papers
No similar papers found.
