References & Citations
Mathematics > Numerical Analysis
Title: Greed Works: An Improved Analysis of Sampling Kaczmarz-Motzkin
(Submitted on 7 Dec 2019 (v1), last revised 24 Jul 2020 (this version, v2))
Abstract: Stochastic iterative algorithms have gained recent interest in machine learning and signal processing for solving large-scale systems of equations, $Ax=b$. One such example is the Randomized Kaczmarz (RK) algorithm, which acts only on single rows of the matrix $A$ at a time. While RK randomly selects a row of $A$ to work with, Motzkin's Method (MM) employs a greedy row selection. Connections between the two algorithms resulted in the Sampling Kaczmarz-Motzkin (SKM) algorithm which samples a random subset of $\beta$ rows of $A$ and then greedily selects the best row of the subset. Despite their variable computational costs, all three algorithms have been proven to have the same theoretical upper bound on the convergence rate. In this work, an improved analysis of the range of random (RK) to greedy (MM) methods is presented. This analysis improves upon previous known convergence bounds for SKM, capturing the benefit of partially greedy selection schemes. This work also further generalizes previous known results, removing the theoretical assumptions that $\beta$ must be fixed at every iteration and that $A$ must have normalized rows.
Submission history
From: Anna Ma [view email][v1] Sat, 7 Dec 2019 18:56:59 GMT (890kb,D)
[v2] Fri, 24 Jul 2020 19:14:21 GMT (1391kb,D)
Link back to: arXiv, form interface, contact.