Current browse context:
math.FA
Change to browse by:
References & Citations
Mathematics > Functional Analysis
Title: Approximation of Smoothness Classes by Deep Rectifier Networks
(Submitted on 30 Jul 2020 (v1), last revised 29 Oct 2021 (this version, v2))
Abstract: We consider approximation rates of sparsely connected deep rectified linear unit (ReLU) and rectified power unit (RePU) neural networks for functions in Besov spaces $B^\alpha_{q}(L^p)$ in arbitrary dimension $d$, on general domains. We show that \alert{deep rectifier} networks with a fixed activation function attain optimal or near to optimal approximation rates for functions in the Besov space $B^\alpha_{\tau}(L^\tau)$ on the critical embedding line $1/\tau=\alpha/d+1/p$ for \emph{arbitrary} smoothness order $\alpha>0$. Using interpolation theory, this implies that the entire range of smoothness classes at or above the critical line is (near to) optimally approximated by deep ReLU/RePU networks.
Submission history
From: Anthony Nouy [view email][v1] Thu, 30 Jul 2020 17:58:05 GMT (41kb,D)
[v2] Fri, 29 Oct 2021 14:55:34 GMT (568kb,D)
Link back to: arXiv, form interface, contact.