Current browse context:
math.DS
Change to browse by:
References & Citations
Mathematics > Dynamical Systems
Title: Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regions
(Submitted on 2 May 2016 (v1), last revised 7 Jun 2016 (this version, v2))
Abstract: Given a non-convex twice differentiable cost function f, we prove that the set of initial conditions so that gradient descent converges to saddle points where \nabla^2 f has at least one strictly negative eigenvalue has (Lebesgue) measure zero, even for cost functions f with non-isolated critical points, answering an open question in [Lee, Simchowitz, Jordan, Recht, COLT2016]. Moreover, this result extends to forward-invariant convex subspaces, allowing for weak (non-globally Lipschitz) smoothness assumptions. Finally, we produce an upper bound on the allowable step-size.
Submission history
From: Ioannis Panageas [view email][v1] Mon, 2 May 2016 09:34:19 GMT (8kb)
[v2] Tue, 7 Jun 2016 07:49:13 GMT (173kb,D)
Link back to: arXiv, form interface, contact.