We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: IHT dies hard: Provable accelerated Iterative Hard Thresholding

Abstract: We study --both in theory and practice-- the use of momentum motions in classic iterative hard thresholding (IHT) methods. By simply modifying plain IHT, we investigate its convergence behavior on convex optimization criteria with non-convex constraints, under standard assumptions. In diverse scenaria, we observe that acceleration in IHT leads to significant improvements, compared to state of the art projected gradient descent and Frank-Wolfe variants. As a byproduct of our inspection, we study the impact of selecting the momentum parameter: similar to convex settings, two modes of behavior are observed --"rippling" and linear-- depending on the level of momentum.
Comments: accepted to AISTATS 2018
Subjects: Optimization and Control (math.OC); Data Structures and Algorithms (cs.DS); Machine Learning (cs.LG); Numerical Analysis (math.NA); Machine Learning (stat.ML)
Cite as: arXiv:1712.09379 [math.OC]
  (or arXiv:1712.09379v2 [math.OC] for this version)

Submission history

From: Anastasios Kyrillidis [view email]
[v1] Tue, 26 Dec 2017 19:40:47 GMT (6873kb,D)
[v2] Fri, 13 Sep 2019 18:01:30 GMT (6917kb,D)

Link back to: arXiv, form interface, contact.