We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.DS

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: The Approximate Duality Gap Technique: A Unified Theory of First-Order Methods

Abstract: We present a general technique for the analysis of first-order methods. The technique relies on the construction of a duality gap for an appropriate approximation of the objective function, where the function approximation improves as the algorithm converges. We show that in continuous time enforcement of an invariant that this approximate duality gap decreases at a certain rate exactly recovers a wide range of first-order continuous-time methods. We characterize the discretization errors incurred by different discretization methods, and show how iteration-complexity-optimal methods for various classes of problems cancel out the discretization error. The techniques are illustrated on various classes of problems -- including convex minimization for Lipschitz-continuous objectives, smooth convex minimization, composite minimization, smooth and strongly convex minimization, solving variational inequalities with monotone operators, and convex-concave saddle-point optimization -- and naturally extend to other settings.
Comments: In SIAM Journal on Optimization. The most recent version corrected a few typos
Subjects: Optimization and Control (math.OC); Data Structures and Algorithms (cs.DS)
DOI: 10.1137/18M1172314
Cite as: arXiv:1712.02485 [math.OC]
  (or arXiv:1712.02485v3 [math.OC] for this version)

Submission history

From: Jelena Diakonikolas [view email]
[v1] Thu, 7 Dec 2017 03:56:01 GMT (28kb)
[v2] Wed, 5 Dec 2018 19:27:41 GMT (46kb,D)
[v3] Wed, 11 Dec 2019 00:54:37 GMT (50kb,D)

Link back to: arXiv, form interface, contact.