Current browse context:
math.OC
Change to browse by:
References & Citations
Mathematics > Optimization and Control
Title: Survey Descent: A Multipoint Generalization of Gradient Descent for Nonsmooth Optimization
(Submitted on 30 Nov 2021 (v1), revised 6 Jan 2022 (this version, v3), latest version 27 Sep 2022 (v5))
Abstract: For strongly convex objectives that are smooth, the classical theory of gradient descent ensures linear convergence relative to the number of gradient evaluations. An analogous nonsmooth theory is challenging: even when the objective is smooth at every iterate, the corresponding local models are unstable, and traditional remedies need unpredictably many cutting planes. We instead propose a multipoint generalization of the gradient descent iteration for local optimization. While designed with general objectives in mind, we are motivated by a "max-of-smooth" model that captures subdifferential dimension at optimality. We prove linear convergence when the objective is itself max-of-smooth, and experiments suggest a more general phenomenon.
Submission history
From: X.Y. Han [view email][v1] Tue, 30 Nov 2021 18:28:17 GMT (1503kb,D)
[v2] Wed, 29 Dec 2021 19:25:52 GMT (1853kb,D)
[v3] Thu, 6 Jan 2022 15:49:23 GMT (1853kb,D)
[v4] Tue, 7 Jun 2022 16:36:04 GMT (3154kb,D)
[v5] Tue, 27 Sep 2022 17:42:31 GMT (3153kb,D)
Link back to: arXiv, form interface, contact.