We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: From the Ravine method to the Nesterov method and vice versa: a dynamical system perspective

Abstract: We revisit the Ravine method of Gelfand and Tsetlin from a dynamical system perspective, study its convergence properties, and highlight its similarities and differences with the Nesterov accelerated gradient method. The two methods are closely related. They can be deduced from each other by reversing the order of the extrapolation and gradient operations in their definitions. They benefit from similar fast convergence of values and convergence of iterates for general convex objective functions. We will also establish the high resolution ODE of the Ravine and Nesterov methods, and reveal an additional geometric damping term driven by the Hessian for both methods. This will allow us to prove fast convergence towards zero of the gradients not only for the Ravine method but also for the Nesterov method for the first time. We also highlight connections to other algorithms stemming from more subtle discretization schemes, and finally describe a Ravine version of the proximal-gradient algorithms for general structured smooth + non-smooth convex optimization problems.
Subjects: Optimization and Control (math.OC); Numerical Analysis (math.NA)
MSC classes: 37N40, 46N10, 49M30, 65B99, 65K05, 65K10, 90B50, 90C25
Cite as: arXiv:2201.11643 [math.OC]
  (or arXiv:2201.11643v2 [math.OC] for this version)

Submission history

From: Jalal Fadili [view email]
[v1] Thu, 27 Jan 2022 17:05:38 GMT (162kb,D)
[v2] Tue, 1 Feb 2022 16:34:05 GMT (163kb,D)

Link back to: arXiv, form interface, contact.