We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Computer Science > Machine Learning

Title: Two-layer neural networks with values in a Banach space

Authors: Yury Korolev
Abstract: We study two-layer neural networks whose domain and range are Banach spaces with separable preduals. In addition, we assume that the image space is equipped with a partial order, i.e. it is a Riesz space. As the nonlinearity we choose the lattice operation of taking the positive part; in case of $\mathbb R^d$-valued neural networks this corresponds to the ReLU activation function. We prove inverse and direct approximation theorems with Monte-Carlo rates for a certain class of functions, extending existing results for the finite-dimensional case. In the second part of the paper, we study, from the regularisation theory viewpoint, the problem of finding optimal representations of such functions via signed measures on a latent space from a finite number of noisy observations. We discuss regularity conditions known as source conditions and obtain convergence rates in a Bregman distance for the representing measure in the regime when both the noise level goes to zero and the number of samples goes to infinity at appropriate rates.
Subjects: Machine Learning (cs.LG); Functional Analysis (math.FA); Numerical Analysis (math.NA); Probability (math.PR)
MSC classes: 68Q32, 68T07, 46E40, 41A65, 65J22
Cite as: arXiv:2105.02095 [cs.LG]
  (or arXiv:2105.02095v5 [cs.LG] for this version)

Submission history

From: Yury Korolev [view email]
[v1] Wed, 5 May 2021 14:54:24 GMT (57kb,D)
[v2] Fri, 13 May 2022 14:02:13 GMT (47kb,D)
[v3] Thu, 18 Aug 2022 14:12:28 GMT (48kb,D)
[v4] Fri, 2 Sep 2022 09:45:48 GMT (48kb,D)
[v5] Wed, 9 Nov 2022 09:46:54 GMT (52kb,D)

Link back to: arXiv, form interface, contact.