Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Approximation and convergence of GANs training: an SDE approach
(Submitted on 3 Jun 2020 (v1), revised 15 Jun 2020 (this version, v2), latest version 12 Feb 2022 (v5))
Abstract: Generative adversarial networks (GANs) have enjoyed tremendous empirical successes, and research interest in the theoretical understanding of GANs training process is rapidly growing, especially for its evolution and convergence analysis. This paper establishes approximations, with precise error bound analysis, for the training of GANs under stochastic gradient algorithms (SGAs). The approximations are in the form of coupled stochastic differential equations (SDEs). The analysis of the SDEs and the associated invariant measures yields conditions for the convergence of GANs training. Further analysis of the invariant measure for the coupled SDEs gives rise to a fluctuation-dissipation relations (FDRs) for GANs, revealing the trade-off of the loss landscape between the generator and the discriminator and providing guidance for learning rate scheduling.
Submission history
From: Haoyang Cao [view email][v1] Wed, 3 Jun 2020 05:08:21 GMT (15kb)
[v2] Mon, 15 Jun 2020 05:14:19 GMT (19kb)
[v3] Thu, 13 Aug 2020 02:39:57 GMT (20kb)
[v4] Thu, 19 Nov 2020 16:10:47 GMT (27kb)
[v5] Sat, 12 Feb 2022 17:17:54 GMT (356kb)
Link back to: arXiv, form interface, contact.