Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Riemannian stochastic quasi-Newton algorithm with variance reduction and its convergence analysis
(Submitted on 15 Mar 2017 (v1), last revised 16 Sep 2017 (this version, v3))
Abstract: Stochastic variance reduction algorithms have recently become popular for minimizing the average of a large, but finite number of loss functions. The present paper proposes a Riemannian stochastic quasi-Newton algorithm with variance reduction (R-SQN-VR). The key challenges of averaging, adding, and subtracting multiple gradients are addressed with notions of retraction and vector transport. We present convergence analyses of R-SQN-VR on both non-convex and retraction-convex functions under retraction and vector transport operators. The proposed algorithm is evaluated on the Karcher mean computation on the symmetric positive-definite manifold and the low-rank matrix completion on the Grassmann manifold. In all cases, the proposed algorithm outperforms the state-of-the-art Riemannian batch and stochastic gradient algorithms.
Submission history
From: Hiroyuki Kasai [view email][v1] Wed, 15 Mar 2017 02:34:39 GMT (2113kb,D)
[v2] Wed, 12 Apr 2017 15:52:38 GMT (2114kb,D)
[v3] Sat, 16 Sep 2017 09:47:22 GMT (1045kb,D)
Link back to: arXiv, form interface, contact.