We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Flexible Modification of Gauss-Newton Method and Its Stochastic Extension

Abstract: This work presents a novel version of recently developed Gauss-Newton method for solving systems of nonlinear equations, based on upper bound of solution residual and quadratic regularization ideas. We obtained for such method global convergence bounds and under natural non-degeneracy assumptions we present local quadratic convergence results. We developed stochastic optimization algorithms for presented Gauss-Newton method and justified sub-linear and linear convergence rates for these algorithms using weak growth condition (WGC) and Polyak-Lojasiewicz (PL) inequality. We show that Gauss-Newton method in stochastic setting can effectively find solution under WGC and PL condition matching convergence rate of the deterministic optimization method. The suggested method unifies most practically used Gauss-Newton method modifications and can easily interpolate between them providing flexible and convenient method easily implementable using standard techniques of convex optimization.
Comments: (in Russian, 105 pages)
Subjects: Optimization and Control (math.OC)
Cite as: arXiv:2102.00810 [math.OC]
  (or arXiv:2102.00810v2 [math.OC] for this version)

Submission history

From: Nikita Yudin [view email]
[v1] Mon, 1 Feb 2021 12:58:36 GMT (99kb)
[v2] Mon, 3 May 2021 00:45:20 GMT (100kb)

Link back to: arXiv, form interface, contact.