Current browse context:
math.OC
Change to browse by:
References & Citations
Mathematics > Optimization and Control
Title: A Contraction Theory Approach to Optimization Algorithms from Acceleration Flows
(Submitted on 18 May 2021 (v1), last revised 31 Jan 2022 (this version, v3))
Abstract: Much recent interest has focused on the design of optimization algorithms from the discretization of an associated optimization flow, i.e., a system of differential equations (ODEs) whose trajectories solve an associated optimization problem. Such a design approach poses an important problem: how to find a principled methodology to design and discretize appropriate ODEs. This paper aims to provide a solution to this problem through the use of contraction theory. We first introduce general mathematical results that explain how contraction theory guarantees the stability of the implicit and explicit Euler integration methods. Then, we propose a novel system of ODEs, namely the Accelerated-Contracting-Nesterov flow, and use contraction theory to establish it is an optimization flow with exponential convergence rate, from which the linear convergence rate of its associated optimization algorithm is immediately established. Remarkably, a simple explicit Euler discretization of this flow corresponds to the Nesterov acceleration method. Finally, we present how our approach leads to performance guarantees in the design of optimization algorithms for time-varying optimization problems.
Submission history
From: Pedro Cisneros-Velarde [view email][v1] Tue, 18 May 2021 21:11:37 GMT (35kb)
[v2] Wed, 13 Oct 2021 18:47:26 GMT (37kb)
[v3] Mon, 31 Jan 2022 20:39:07 GMT (20kb)
Link back to: arXiv, form interface, contact.