We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Tailoring Gradient Methods for Differentially-Private Distributed Optimization

Abstract: Decentralized optimization is gaining increased traction due to its widespread applications in large-scale machine learning and multi-agent systems. The same mechanism that enables its success, i.e., information sharing among participating agents, however, also leads to the disclosure of individual agents' private information, which is unacceptable when sensitive data are involved. As differential privacy is becoming a de facto standard for privacy preservation, recently results have emerged integrating differential privacy with distributed optimization. However, directly incorporating differential privacy design in existing distributed optimization approaches significantly compromises optimization accuracy. In this paper, we propose to redesign and tailor gradient methods for differentially-private distributed optimization, and propose two differential-privacy oriented gradient methods that can ensure both rigorous epsilon-differential privacy and optimality. The first algorithm is based on static-consensus based gradient methods, and the second algorithm is based on dynamic-consensus (gradient-tracking) based distributed optimization methods and, hence, is applicable to general directed interaction graph topologies. Both algorithms can simultaneously ensure almost sure convergence to an optimal solution and a finite privacy budget, even when the number of iterations goes to infinity. To our knowledge, this is the first time that both goals are achieved simultaneously. Numerical simulations using a distributed estimation problem and experimental results on a benchmark dataset confirm the effectiveness of the proposed approaches.
Subjects: Optimization and Control (math.OC); Distributed, Parallel, and Cluster Computing (cs.DC)
Cite as: arXiv:2202.01113 [math.OC]
  (or arXiv:2202.01113v5 [math.OC] for this version)

Submission history

From: Yongqiang Wang [view email]
[v1] Wed, 2 Feb 2022 16:20:07 GMT (170kb)
[v2] Fri, 1 Apr 2022 16:36:28 GMT (368kb)
[v3] Sun, 29 May 2022 01:30:37 GMT (241kb)
[v4] Sat, 5 Nov 2022 22:43:54 GMT (228kb)
[v5] Mon, 28 Nov 2022 22:02:50 GMT (228kb)

Link back to: arXiv, form interface, contact.