References & Citations
Mathematics > Optimization and Control
Title: A Decentralized Primal-Dual Framework for Non-convex Smooth Consensus Optimization
(Submitted on 23 Jul 2021 (v1), last revised 3 Feb 2023 (this version, v3))
Abstract: In this work, we introduce ADAPD, $\textbf{A}$ $\textbf{D}$ecentr$\textbf{A}$lized $\textbf{P}$rimal-$\textbf{D}$ual algorithmic framework for solving non-convex and smooth consensus optimization problems over a network of distributed agents. The proposed framework relies on a novel problem formulation that elicits ADMM-type updates, where each agent first inexactly solves a local strongly convex subproblem with any method of its choice and then performs a neighbor communication to update a set of dual variables. We present two variants that allow for a single gradient step for the primal updates or multiple communications for the dual updates, to exploit the tradeoff between the per-iteration cost and the number of iterations. When multiple communications are performed, ADAPD can achieve theoretically optimal communication complexity results for non-convex and smooth consensus problems. Numerical experiments on several applications, including a deep-learning one, demonstrate the superiority of ADAPD over several popularly used decentralized methods.
Submission history
From: Gabriel Mancino-Ball [view email][v1] Fri, 23 Jul 2021 16:00:39 GMT (4408kb,D)
[v2] Mon, 14 Feb 2022 16:19:29 GMT (3137kb,D)
[v3] Fri, 3 Feb 2023 20:47:11 GMT (3493kb,D)
Link back to: arXiv, form interface, contact.