We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Acceleration of Frank-Wolfe algorithms with open loop step-sizes

Abstract: Frank-Wolfe algorithms (FW) are popular first-order methods to solve convex constrained optimization problems that rely on a linear minimization oracle instead of potentially expensive projection-like oracles. Many works have identified accelerated convergence rates under various structural assumptions on the optimization problem and for specific FW variants when using line search or short-step, requiring feedback from the objective function. Little is known about accelerated convergence regimes when utilizing open loop step-size rules, a.k.a. FW with pre-determined step-sizes, which are algorithmically extremely simple and stable. Not only is FW with open loop step-size rules not always subject to the same convergence rate lower bounds as FW with line search or short-step, but in some specific cases, such as kernel herding in infinite-dimensions, it is observed that FW with open loop step-size rules leads to faster convergence as opposed to FW with line search or short-step. We propose a partial answer to this open problem in kernel herding, characterize a general setting for which FW with open loop step-size rules converges non-asymptotically faster than with line search or short-step, and derive several accelerated convergence results for FW (and two of its variants) with open loop step-size rules. Finally, our numerical experiments highlight potential gaps in our current understanding of the FW method in general.
Subjects: Optimization and Control (math.OC)
Cite as: arXiv:2205.12838 [math.OC]
  (or arXiv:2205.12838v2 [math.OC] for this version)

Submission history

From: Elias Wirth [view email]
[v1] Wed, 25 May 2022 15:06:46 GMT (3548kb,D)
[v2] Fri, 27 May 2022 10:02:13 GMT (3548kb,D)
[v3] Sun, 5 Jun 2022 14:35:11 GMT (3548kb,D)
[v4] Fri, 17 Jun 2022 09:36:36 GMT (3550kb,D)
[v5] Tue, 21 Jun 2022 07:39:05 GMT (3550kb,D)

Link back to: arXiv, form interface, contact.