We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: A nonlinear conjugate gradient method with complexity guarantees and its application to nonconvex regression

Abstract: Nonlinear conjugate gradients are among the most popular techniques for solving continuous optimization problems. Although these schemes have long been studied from a global convergence standpoint, their worst-case complexity properties have yet to be fully understood, especially in the nonconvex setting. In particular, it is unclear whether such methods possess better guarantees than first-order methods such as gradient descent. On the other hand, recent results have shown good performance of standard nonlinear conjugate gradient methods on nonconvex problems, even when compared with methods endowed with the best known complexity guarantees.
In this paper, we propose a nonlinear conjugate gradient method based on a simple line-search paradigm and a modified restart condition. These two ingredients allow for monitoring the properties of the search directions, which is instrumental in obtaining complexity guarantees. Our complexity results illustrate the possible discrepancy between nonlinear conjugate gradient methods and classical gradient descent. Numerical experiments on nonconvex robust regression problems suggest that the restarting condition has little impact over the practical behavior.
Subjects: Optimization and Control (math.OC)
MSC classes: 90C60, 65K05, 49M05
Cite as: arXiv:2201.08568 [math.OC]
  (or arXiv:2201.08568v1 [math.OC] for this version)

Submission history

From: Clément W. Royer [view email]
[v1] Fri, 21 Jan 2022 07:03:38 GMT (126kb,D)

Link back to: arXiv, form interface, contact.