Current browse context:
math.OC
Change to browse by:
References & Citations
Mathematics > Optimization and Control
Title: Distributed Adaptive Newton Methods with Global Superlinear Convergence
(Submitted on 18 Feb 2020 (v1), last revised 14 Jan 2022 (this version, v3))
Abstract: This paper considers the distributed optimization problem where each node of a peer-to-peer network minimizes a finite sum of objective functions by communicating with its neighboring nodes. In sharp contrast to the existing literature where the fastest distributed algorithms converge either with a global linear or a local superlinear rate, we propose a distributed adaptive Newton (DAN) algorithm with a global quadratic convergence rate. Our key idea lies in the design of a finite-time set-consensus method with Polyak's adaptive stepsize. Moreover, we introduce a low-rank matrix approximation (LA) technique to compress the innovation of Hessian matrix so that each node only needs to transmit message of dimension $\mathcal{O}(p)$ (where $p$ is the dimension of decision vectors) per iteration, which is essentially the same as that of first-order methods. Nevertheless, the resulting DAN-LA converges to an optimal solution with a global superlinear rate. Numerical experiments on logistic regression problems are conducted to validate their advantages over existing methods.
Submission history
From: Jiaqi Zhang [view email][v1] Tue, 18 Feb 2020 05:29:30 GMT (896kb,D)
[v2] Fri, 8 May 2020 13:43:48 GMT (896kb,D)
[v3] Fri, 14 Jan 2022 13:50:59 GMT (935kb,D)
Link back to: arXiv, form interface, contact.