Current browse context:
math.OC
Change to browse by:
References & Citations
Mathematics > Optimization and Control
Title: Less than a Single Pass: Stochastically Controlled Stochastic Gradient Method
(Submitted on 12 Sep 2016 (v1), last revised 16 May 2019 (this version, v3))
Abstract: We develop and analyze a procedure for gradient-based optimization that we refer to as stochastically controlled stochastic gradient (SCSG). As a member of the SVRG family of algorithms, SCSG makes use of gradient estimates at two scales, with the number of updates at the faster scale being governed by a geometric random variable. Unlike most existing algorithms in this family, both the computation cost and the communication cost of SCSG do not necessarily scale linearly with the sample size $n$; indeed, these costs are independent of $n$ when the target accuracy is low. An experimental evaluation on real datasets confirms the effectiveness of SCSG.
Submission history
From: Lihua Lei [view email][v1] Mon, 12 Sep 2016 03:35:29 GMT (44kb,D)
[v2] Mon, 3 Jul 2017 00:25:50 GMT (570kb,D)
[v3] Thu, 16 May 2019 04:18:36 GMT (557kb,D)
Link back to: arXiv, form interface, contact.