Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: On the capacity of deep generative networks for approximating distributions
(Submitted on 29 Jan 2021 (this version), latest version 17 Oct 2021 (v3))
Abstract: We study the efficacy and efficiency of deep generative networks for approximating probability distributions. We prove that neural networks can transform a one-dimensional source distribution to a distribution that is arbitrarily close to a high-dimensional target distribution in Wasserstein distances. Upper bounds of the approximation error are obtained in terms of neural networks' width and depth. It is shown that the approximation error grows at most linearly on the ambient dimension and that the approximation order only depends on the intrinsic dimension of the target distribution. On the contrary, when $f$-divergences are used as metrics of distributions, the approximation property is different. We prove that in order to approximate the target distribution in $f$-divergences, the dimension of the source distribution cannot be smaller than the intrinsic dimension of the target distribution. Therefore, $f$-divergences are less adequate than Waserstein distances as metrics of distributions for generating samples.
Submission history
From: Yunfei Yang [view email][v1] Fri, 29 Jan 2021 01:45:02 GMT (55kb,D)
[v2] Thu, 13 May 2021 07:42:34 GMT (145kb,D)
[v3] Sun, 17 Oct 2021 03:16:09 GMT (146kb,D)
Link back to: arXiv, form interface, contact.