References & Citations
Mathematics > Optimization and Control
Title: On the convergence of stochastic primal-dual hybrid gradient
(Submitted on 2 Nov 2019 (v1), last revised 21 Jun 2022 (this version, v3))
Abstract: In this paper, we analyze the recently proposed stochastic primal-dual hybrid gradient (SPDHG) algorithm and provide new theoretical results. In particular, we prove almost sure convergence of the iterates to a solution with convexity and linear convergence with further structure, using standard step sizes independent of strong convexity or other regularity constants. In the general convex case, we also prove the $\mathcal{O}(1/k)$ convergence rate for the ergodic sequence, on expected primal-dual gap function. Our assumption for linear convergence is metric subregularity, which is satisfied for strongly convex-strongly concave problems in addition to many nonsmooth and/or nonstrongly convex problems, such as linear programs, Lasso, and support vector machines. We also provide numerical evidence showing that SPDHG with standard step sizes shows a competitive practical performance against its specialized strongly convex variant SPDHG-$\mu$ and other state-of-the-art algorithms including variance reduction methods.
Submission history
From: Ahmet Alacaoglu [view email][v1] Sat, 2 Nov 2019 23:53:06 GMT (114kb,D)
[v2] Wed, 8 Jan 2020 18:01:18 GMT (119kb,D)
[v3] Tue, 21 Jun 2022 18:50:42 GMT (450kb,D)
Link back to: arXiv, form interface, contact.