Current browse context:
math.OC
Change to browse by:
References & Citations
Mathematics > Optimization and Control
Title: Linearly convergent stochastic heavy ball method for minimizing generalization error
(Submitted on 30 Oct 2017 (v1), last revised 23 Dec 2017 (this version, v2))
Abstract: In this work we establish the first linear convergence result for the stochastic heavy ball method. The method performs SGD steps with a fixed stepsize, amended by a heavy ball momentum term. In the analysis, we focus on minimizing the expected loss and not on finite-sum minimization, which is typically a much harder problem. While in the analysis we constrain ourselves to quadratic loss, the overall objective is not necessarily strongly convex.
Submission history
From: Nicolas Loizou [view email][v1] Mon, 30 Oct 2017 01:49:34 GMT (94kb,D)
[v2] Sat, 23 Dec 2017 02:18:06 GMT (92kb,D)
Link back to: arXiv, form interface, contact.