We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Stochastic Mirror Descent for Convex Optimization with Consensus Constraints

Abstract: The mirror descent algorithm is known to be effective in applications where it is beneficial to adapt the mirror map to the underlying geometry of the optimization model. However, the effect of mirror maps on the geometry of distributed optimization problems has not been previously addressed. In this paper we propose and study exact distributed mirror descent algorithms in continuous-time under additive noise and present the settings that enable linear convergence rates. Our analysis draws motivation from the augmented Lagrangian and its relation to gradient tracking. To further explore the benefits of mirror maps in a distributed setting we present a preconditioned variant of our algorithm with an additional mirror map over the Lagrangian dual variables. This allows our method to adapt to the geometry of the consensus manifold and leads to faster convergence. We illustrate the performance of the algorithms in convex settings both with and without constraints. We also explore their performance numerically in a non-convex application with neural networks.
Subjects: Optimization and Control (math.OC)
MSC classes: 65K10, 68Q87, 60G07, 68W15
Cite as: arXiv:2201.08642 [math.OC]
  (or arXiv:2201.08642v2 [math.OC] for this version)

Submission history

From: Anastasia Borovykh [view email]
[v1] Fri, 21 Jan 2022 11:03:19 GMT (5751kb,D)
[v2] Wed, 2 Feb 2022 19:58:04 GMT (5752kb,D)

Link back to: arXiv, form interface, contact.