Current browse context:
math.OC
Change to browse by:
References & Citations
Mathematics > Optimization and Control
Title: Stochastic Subgradient Descent on a Generic Definable Function Converges to a Minimizer
(Submitted on 6 Sep 2021 (v1), last revised 10 Feb 2022 (this version, v3))
Abstract: It was previously shown by Davis and Drusvyatskiy that every Clarke critical point of a generic, semialgebraic (and more generally definable in an o-minimal structure), weakly convex function is lying on an active manifold and is either a local minimum or an active strict saddle. In the first part of this work, we show that when the weak convexity assumption fails a third type of point appears: a sharply repulsive critical point. Moreover, we show that the corresponding active manifolds satisfy the Verdier and the angle conditions which were introduced by us in our previous work. In the second part of this work, we show that, under a density-like assumption on the perturbation sequence, the stochastic subgradient descent (SGD) avoids sharply repulsive critical points with probability one. We show that such a density-like assumption could be obtained upon adding a small random perturbation (e.g. a nondegenerate Gaussian) at each iteration of the algorithm. These results, combined with our previous work on the avoidance of active strict saddles, show that the SGD on a generic definable (e.g. semialgebraic) function converges to a local minimum.
Submission history
From: Sholom Schechtman [view email][v1] Mon, 6 Sep 2021 13:35:56 GMT (50kb)
[v2] Fri, 1 Oct 2021 19:07:02 GMT (300kb)
[v3] Thu, 10 Feb 2022 22:23:56 GMT (0kb,I)
Link back to: arXiv, form interface, contact.