Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Stochastic Cubic Regularization for Fast Nonconvex Optimization
(Submitted on 8 Nov 2017 (v1), last revised 5 Dec 2017 (this version, v2))
Abstract: This paper proposes a stochastic variant of a classic algorithm---the cubic-regularized Newton method [Nesterov and Polyak 2006]. The proposed algorithm efficiently escapes saddle points and finds approximate local minima for general smooth, nonconvex functions in only $\mathcal{\tilde{O}}(\epsilon^{-3.5})$ stochastic gradient and stochastic Hessian-vector product evaluations. The latter can be computed as efficiently as stochastic gradients. This improves upon the $\mathcal{\tilde{O}}(\epsilon^{-4})$ rate of stochastic gradient descent. Our rate matches the best-known result for finding local minima without requiring any delicate acceleration or variance-reduction techniques.
Submission history
From: Nilesh Tripuraneni [view email][v1] Wed, 8 Nov 2017 05:39:46 GMT (93kb,D)
[v2] Tue, 5 Dec 2017 20:40:44 GMT (452kb,D)
Link back to: arXiv, form interface, contact.