Current browse context:
math.OC
Change to browse by:
References & Citations
Mathematics > Optimization and Control
Title: Tracking the gradients using the Hessian: A new look at variance reducing stochastic methods
(Submitted on 20 Oct 2017 (v1), last revised 31 Mar 2018 (this version, v3))
Abstract: Our goal is to improve variance reducing stochastic methods through better control variates. We first propose a modification of SVRG which uses the Hessian to track gradients over time, rather than to recondition, increasing the correlation of the control variates and leading to faster theoretical convergence close to the optimum. We then propose accurate and computationally efficient approximations to the Hessian, both using a diagonal and a low-rank matrix. Finally, we demonstrate the effectiveness of our method on a wide range of problems.
Submission history
From: Robert M. Gower [view email][v1] Fri, 20 Oct 2017 09:31:06 GMT (2396kb,D)
[v2] Tue, 27 Feb 2018 22:15:22 GMT (3031kb,D)
[v3] Sat, 31 Mar 2018 19:14:10 GMT (3031kb,D)
Link back to: arXiv, form interface, contact.