Current browse context:
math.OC
Change to browse by:
References & Citations
Mathematics > Optimization and Control
Title: Dualize, Split, Randomize: Fast Nonsmooth Optimization Algorithms
(Submitted on 3 Apr 2020 (this version), latest version 26 Jul 2022 (v4))
Abstract: We introduce a new primal-dual algorithm for minimizing the sum of three convex functions, each of which has its own oracle. Namely, the first one is differentiable, smooth and possibly stochastic, the second is proximable, and the last one is a composition of a proximable function with a linear map. Our theory covers several settings that are not tackled by any existing algorithm; we illustrate their importance with real-world applications. By leveraging variance reduction, we obtain convergence with linear rates under strong convexity and fast sublinear convergence under convexity assumptions. The proposed theory is simple and unified by the umbrella of stochastic Davis-Yin splitting, which we design in this work. Finally, we illustrate the efficiency of our method through numerical experiments.
Submission history
From: Laurent Condat [view email][v1] Fri, 3 Apr 2020 10:48:01 GMT (118kb,D)
[v2] Fri, 12 Jun 2020 09:18:31 GMT (125kb,D)
[v3] Fri, 18 Mar 2022 06:24:05 GMT (72kb,D)
[v4] Tue, 26 Jul 2022 11:21:34 GMT (73kb,D)
Link back to: arXiv, form interface, contact.