We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Convergence and Convergence Rate of Stochastic Gradient Search in the Case of Multiple and Non-Isolated Extrema

Abstract: The asymptotic behavior of stochastic gradient algorithms is studied. Relying on results from differential geometry (Lojasiewicz gradient inequality), the single limit-point convergence of the algorithm iterates is demonstrated and relatively tight bounds on the convergence rate are derived. In sharp contrast to the existing asymptotic results, the new results presented here allow the objective function to have multiple and non-isolated minima. The new results also offer new insights into the asymptotic properties of several classes of recursive algorithms which are routinely used in engineering, statistics, machine learning and operations research.
Subjects: Optimization and Control (math.OC); Statistics Theory (math.ST)
MSC classes: 62L20, 90C15, 93E12, 93E35
Cite as: arXiv:0907.1020 [math.OC]
  (or arXiv:0907.1020v5 [math.OC] for this version)

Submission history

From: Vladislav Tadić B [view email]
[v1] Mon, 6 Jul 2009 15:42:21 GMT (34kb)
[v2] Tue, 7 Jul 2009 17:27:40 GMT (34kb)
[v3] Fri, 17 Jul 2009 15:46:50 GMT (35kb)
[v4] Thu, 21 Jul 2011 16:08:55 GMT (52kb)
[v5] Tue, 17 Sep 2013 20:01:30 GMT (67kb)

Link back to: arXiv, form interface, contact.