Current browse context:
stat
Change to browse by:
References & Citations
Mathematics > Functional Analysis
Title: Kolmogorov Width Decay and Poor Approximators in Machine Learning: Shallow Neural Networks, Random Feature Models and Neural Tangent Kernels
(Submitted on 21 May 2020 (v1), last revised 2 Oct 2020 (this version, v2))
Abstract: We establish a scale separation of Kolmogorov width type between subspaces of a given Banach space under the condition that a sequence of linear maps converges much faster on one of the subspaces. The general technique is then applied to show that reproducing kernel Hilbert spaces are poor $L^2$-approximators for the class of two-layer neural networks in high dimension, and that multi-layer networks with small path norm are poor approximators for certain Lipschitz functions, also in the $L^2$-topology.
Submission history
From: Stephan Wojtowytsch [view email][v1] Thu, 21 May 2020 17:40:38 GMT (25kb)
[v2] Fri, 2 Oct 2020 05:33:48 GMT (26kb)
Link back to: arXiv, form interface, contact.