Current browse context:
math.OC
Change to browse by:
References & Citations
Mathematics > Optimization and Control
Title: Cyclic Coordinate Update Algorithms for Fixed-Point Problems: Analysis and Applications
(Submitted on 8 Nov 2016 (v1), last revised 2 Mar 2017 (this version, v2))
Abstract: Many problems reduce to the fixed-point problem of solving $x=T(x)$. To this problem, we apply the coordinate-update algorithms, which update only one or a few components of $x$ at each step. When each update is cheap, these algorithms are faster than the full fixed-point iteration (which updates all the components).
In this paper, we focus on the coordinate-update algorithms based on the cyclic selection rules, where the ordering of coordinates in each cycle is arbitrary. These algorithms are fast, but their convergence is unknown in the fixed-point setting.
When $T$ is a nonexpansive operator and has a fixed point, we show that the sequence of coordinate-update iterates converges to a fixed point under proper step sizes. This result applies to the primal-dual coordinate-update algorithms, which have applications to optimization problems with nonseparable nonsmooth objectives, as well as global linear constraints.
Numerically, we apply coordinate-update algorithms with the cyclic, shuffled cyclic, and random selection rules to $\ell_1$ robust least squares, a CT image reconstruction problem, as well as nonnegative matrix factorization. They converge much faster than the standard fixed-point iteration. Among the three rules, cyclic and shuffled cyclic rules are overall faster than the random rule.
Submission history
From: Tianyu Wu [view email][v1] Tue, 8 Nov 2016 09:55:00 GMT (354kb,D)
[v2] Thu, 2 Mar 2017 23:20:39 GMT (356kb,D)
Link back to: arXiv, form interface, contact.