We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Can Shallow Neural Networks Beat the Curse of Dimensionality? A mean field training perspective

Abstract: We prove that the gradient descent training of a two-layer neural network on empirical or population risk may not decrease population risk at an order faster than $t^{-4/(d-2)}$ under mean field scaling. Thus gradient descent training for fitting reasonably smooth, but truly high-dimensional data may be subject to the curse of dimensionality. We present numerical evidence that gradient descent training with general Lipschitz target functions becomes slower and slower as the dimension increases, but converges at approximately the same rate in all dimensions when the target function lies in the natural function space for two-layer ReLU networks.
Comments: 5 figures
Subjects: Machine Learning (cs.LG); Analysis of PDEs (math.AP); Machine Learning (stat.ML)
MSC classes: 68T07, 49Q22, 68W25
Cite as: arXiv:2005.10815 [cs.LG]
  (or arXiv:2005.10815v1 [cs.LG] for this version)

Submission history

From: Stephan Wojtowytsch [view email]
[v1] Thu, 21 May 2020 17:50:15 GMT (575kb,D)

Link back to: arXiv, form interface, contact.