Current browse context:
stat
Change to browse by:
References & Citations
Mathematics > Optimization and Control
Title: FLAG n' FLARE: Fast Linearly-Coupled Adaptive Gradient Methods
(Submitted on 26 May 2016 (v1), last revised 11 Nov 2017 (this version, v2))
Abstract: We consider first order gradient methods for effectively optimizing a composite objective in the form of a sum of smooth and, potentially, non-smooth functions. We present accelerated and adaptive gradient methods, called FLAG and FLARE, which can offer the best of both worlds. They can achieve the optimal convergence rate by attaining the optimal first-order oracle complexity for smooth convex optimization. Additionally, they can adaptively and non-uniformly re-scale the gradient direction to adapt to the limited curvature available and conform to the geometry of the domain. We show theoretically and empirically that, through the compounding effects of acceleration and adaptivity, FLAG and FLARE can be highly effective for many data fitting and machine learning applications.
Submission history
From: Xiang Cheng [view email][v1] Thu, 26 May 2016 00:24:37 GMT (19kb)
[v2] Sat, 11 Nov 2017 20:20:17 GMT (2054kb,D)
Link back to: arXiv, form interface, contact.