Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: GDA-AM: On the effectiveness of solving minimax optimization via Anderson Acceleration
(Submitted on 6 Oct 2021 (v1), last revised 29 Jun 2022 (this version, v3))
Abstract: Many modern machine learning algorithms such as generative adversarial networks (GANs) and adversarial training can be formulated as minimax optimization. Gradient descent ascent (GDA) is the most commonly used algorithm due to its simplicity. However, GDA can converge to non-optimal minimax points. We propose a new minimax optimization framework, GDA-AM, that views the GDAdynamics as a fixed-point iteration and solves it using Anderson Mixing to con-verge to the local minimax. It addresses the diverging issue of simultaneous GDAand accelerates the convergence of alternating GDA. We show theoretically that the algorithm can achieve global convergence for bilinear problems under mild conditions. We also empirically show that GDA-AMsolves a variety of minimax problems and improves GAN training on several datasets
Submission history
From: Huan He [view email][v1] Wed, 6 Oct 2021 02:08:54 GMT (20373kb,D)
[v2] Sun, 28 Nov 2021 23:14:16 GMT (19475kb,D)
[v3] Wed, 29 Jun 2022 18:27:22 GMT (19475kb,D)
Link back to: arXiv, form interface, contact.