We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Augmented Newton Method for Optimization: Global Linear Rate and Momentum Interpretation

Abstract: We propose two variants of Newton method for solving unconstrained minimization problem. Our method leverages optimization techniques such as penalty and augmented Lagrangian method to generate novel variants of the Newton method namely the Penalty Newton method and the Augmented Newton method. In doing so, we recover several well-known existing Newton method variants such as Damped Newton, Levenberg, and Levenberg-Marquardt methods as special cases. Moreover, the proposed Augmented Newton method can be interpreted as Newton method with adaptive heavy ball momentum. We provide global convergence results for the proposed methods under mild assumptions that hold for a wide variety of problems. The proposed methods can be sought as the penalty and augmented extensions of the results obtained by Karimireddy et. al [24].
Subjects: Optimization and Control (math.OC); Machine Learning (cs.LG); Numerical Analysis (math.NA)
MSC classes: 49M15, 49M37, 58C15, 65K05, 65K10, 65Y20, 68Q25, 90C06, 90C30, 90C51
Cite as: arXiv:2205.11033 [math.OC]
  (or arXiv:2205.11033v1 [math.OC] for this version)

Submission history

From: Md Sarowar Morshed [view email]
[v1] Mon, 23 May 2022 04:34:10 GMT (46kb,D)

Link back to: arXiv, form interface, contact.