We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: On the capacity of deep generative networks for approximating distributions

Abstract: We study the efficacy and efficiency of deep generative networks for approximating probability distributions. We prove that neural networks can transform a low-dimensional source distribution to a distribution that is arbitrarily close to a high-dimensional target distribution, when the closeness are measured by Wasserstein distances and maximum mean discrepancy. Upper bounds of the approximation error are obtained in terms of the width and depth of neural network. Furthermore, it is shown that the approximation error in Wasserstein distance grows at most linearly on the ambient dimension and that the approximation order only depends on the intrinsic dimension of the target distribution. On the contrary, when $f$-divergences are used as metrics of distributions, the approximation property is different. We show that in order to approximate the target distribution in $f$-divergences, the dimension of the source distribution cannot be smaller than the intrinsic dimension of the target distribution.
Subjects: Machine Learning (cs.LG); Probability (math.PR); Statistics Theory (math.ST); Machine Learning (stat.ML)
DOI: 10.1016/j.neunet.2021.10.012
Cite as: arXiv:2101.12353 [cs.LG]
  (or arXiv:2101.12353v3 [cs.LG] for this version)

Submission history

From: Yunfei Yang [view email]
[v1] Fri, 29 Jan 2021 01:45:02 GMT (55kb,D)
[v2] Thu, 13 May 2021 07:42:34 GMT (145kb,D)
[v3] Sun, 17 Oct 2021 03:16:09 GMT (146kb,D)

Link back to: arXiv, form interface, contact.