Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradient
(Submitted on 1 Mar 2017 (this version), latest version 3 Jun 2017 (v2))
Abstract: In this paper, we propose a StochAstic Recursive grAdient algoritHm (SARAH), as well as its practical variant SARAH+, as a novel approach to the finite-sum minimization problems. Different from the vanilla SGD and other modern stochastic methods such as SVRG, S2GD, SAG and SAGA, SARAH admits a simple recursive framework for updating stochastic gradient estimates; when comparing to SAG/SAGA, SARAH does not require a storage of past gradients. The linear convergence rate of SARAH is proven under strong convexity assumption. We also prove a linear convergence rate (in the strongly convex case) for an inner loop of SARAH, the property that SVRG does not possess. Numerical experiments demonstrate the efficiency of our algorithm.
Submission history
From: Lam Nguyen [view email][v1] Wed, 1 Mar 2017 02:08:32 GMT (1589kb)
[v2] Sat, 3 Jun 2017 07:30:20 GMT (1862kb)
Link back to: arXiv, form interface, contact.