We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Accelerated Gradient Descent Learning over Multiple Access Fading Channels

Abstract: We consider a distributed learning problem in a wireless network, consisting of N distributed edge devices and a parameter server (PS). The objective function is a sum of the edge devices' local loss functions, who aim to train a shared model by communicating with the PS over multiple access channels (MAC). This problem has attracted a growing interest in distributed sensing systems, and more recently in federated learning, known as over-the-air computation. In this paper, we develop a novel Accelerated Gradient-descent Multiple Access (AGMA) algorithm that uses momentum-based gradient signals over noisy fading MAC to improve the convergence rate as compared to existing methods. Furthermore, AGMA does not require power control or beamforming to cancel the fading effect, which simplifies the implementation complexity. We analyze AGMA theoretically, and establish a finite-sample bound of the error for both convex and strongly convex loss functions with Lipschitz gradient. For the strongly convex case, we show that AGMA approaches the best-known linear convergence rate as the network increases. For the convex case, we show that AGMA significantly improves the sub-linear convergence rate as compared to existing methods. Finally, we present simulation results using real datasets that demonstrate better performance by AGMA.
Comments: This paper has been accepted for publication in the IEEE Journal on Selected Areas in Communications
Subjects: Machine Learning (cs.LG); Signal Processing (eess.SP)
Cite as: arXiv:2107.12452 [cs.LG]
  (or arXiv:2107.12452v2 [cs.LG] for this version)

Submission history

From: Raz Paul [view email]
[v1] Mon, 26 Jul 2021 19:51:40 GMT (1806kb,D)
[v2] Fri, 8 Oct 2021 11:33:34 GMT (1313kb,D)

Link back to: arXiv, form interface, contact.