We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Randomized Primal-Dual Proximal Block Coordinate Updates

Abstract: In this paper we propose a randomized primal-dual proximal block coordinate updating framework for a general multi-block convex optimization model with coupled objective function and linear constraints. Assuming mere convexity, we establish its $O(1/t)$ convergence rate in terms of the objective value and feasibility measure. The framework includes several existing algorithms as special cases such as a primal-dual method for bilinear saddle-point problems (PD-S), the proximal Jacobian ADMM (Prox-JADMM) and a randomized variant of the ADMM method for multi-block convex optimization. Our analysis recovers and/or strengthens the convergence properties of several existing algorithms. For example, for PD-S our result leads to the same order of convergence rate without the previously assumed boundedness condition on the constraint sets, and for Prox-JADMM the new result provides convergence rate in terms of the objective value and the feasibility violation. It is well known that the original ADMM may fail to converge when the number of blocks exceeds two. Our result shows that if an appropriate randomization procedure is invoked to select the updating blocks, then a sublinear rate of convergence in expectation can be guaranteed for multi-block ADMM, without assuming any strong convexity. The new approach is also extended to solve problems where only a stochastic approximation of the (sub-)gradient of the objective is available, and we establish an $O(1/\sqrt{t})$ convergence rate of the extended approach for solving stochastic programming.
Comments: convergence rate results are presented in a more explicit way; numerical results are added
Subjects: Optimization and Control (math.OC); Numerical Analysis (math.NA); Machine Learning (stat.ML)
MSC classes: 90C25, 95C06, 68W20
Cite as: arXiv:1605.05969 [math.OC]
  (or arXiv:1605.05969v3 [math.OC] for this version)

Submission history

From: Yangyang Xu [view email]
[v1] Thu, 19 May 2016 14:20:20 GMT (38kb)
[v2] Thu, 26 May 2016 03:57:50 GMT (38kb)
[v3] Tue, 24 Jan 2017 15:43:48 GMT (56kb,D)

Link back to: arXiv, form interface, contact.