Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Communication-Efficient Distributed SVD via Local Power Iterations
(Submitted on 19 Feb 2020 (v1), last revised 4 Jun 2021 (this version, v4))
Abstract: We study distributed computing of the truncated singular value decomposition problem. We develop an algorithm that we call \texttt{LocalPower} for improving communication efficiency. Specifically, we uniformly partition the dataset among $m$ nodes and alternate between multiple (precisely $p$) local power iterations and one global aggregation. In the aggregation, we propose to weight each local eigenvector matrix with orthogonal Procrustes transformation (OPT). As a practical surrogate of OPT, sign-fixing, which uses a diagonal matrix with $\pm 1$ entries as weights, has better computation complexity and stability in experiments. We theoretically show that under certain assumptions \texttt{LocalPower} lowers the required number of communications by a factor of $p$ to reach a constant accuracy. We also show that the strategy of periodically decaying $p$ helps obtain high-precision solutions. We conduct experiments to demonstrate the effectiveness of \texttt{LocalPower}.
Submission history
From: Xiang Li [view email][v1] Wed, 19 Feb 2020 05:58:23 GMT (455kb,D)
[v2] Mon, 5 Apr 2021 12:01:29 GMT (2263kb,D)
[v3] Thu, 3 Jun 2021 15:17:52 GMT (1145kb,D)
[v4] Fri, 4 Jun 2021 08:37:24 GMT (1144kb,D)
Link back to: arXiv, form interface, contact.