Current browse context:
math.OC
Change to browse by:
References & Citations
Mathematics > Optimization and Control
Title: IHT dies hard: Provable accelerated Iterative Hard Thresholding
(Submitted on 26 Dec 2017 (v1), last revised 13 Sep 2019 (this version, v2))
Abstract: We study --both in theory and practice-- the use of momentum motions in classic iterative hard thresholding (IHT) methods. By simply modifying plain IHT, we investigate its convergence behavior on convex optimization criteria with non-convex constraints, under standard assumptions. In diverse scenaria, we observe that acceleration in IHT leads to significant improvements, compared to state of the art projected gradient descent and Frank-Wolfe variants. As a byproduct of our inspection, we study the impact of selecting the momentum parameter: similar to convex settings, two modes of behavior are observed --"rippling" and linear-- depending on the level of momentum.
Submission history
From: Anastasios Kyrillidis [view email][v1] Tue, 26 Dec 2017 19:40:47 GMT (6873kb,D)
[v2] Fri, 13 Sep 2019 18:01:30 GMT (6917kb,D)
Link back to: arXiv, form interface, contact.