We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Zeroth order optimization with orthogonal random directions

Abstract: We propose and analyze a randomized zeroth-order approach based on approximating the exact gradient byfinite differences computed in a set of orthogonal random directions that changes with each iteration. A number ofpreviously proposed methods are recovered as special cases including spherical smoothing, coordinate descent, as wellas discretized gradient descent. Our main contribution is proving convergence guarantees as well as convergence ratesunder different parameter choices and assumptions. In particular, we consider convex objectives, but also possiblynon-convex objectives satisfying the Polyak-{\L}ojasiewicz (PL) condition. Theoretical results are complemented andillustrated by numerical experiments.
Subjects: Optimization and Control (math.OC)
Cite as: arXiv:2107.03941 [math.OC]
  (or arXiv:2107.03941v2 [math.OC] for this version)

Submission history

From: Cesare Molinari [view email]
[v1] Thu, 8 Jul 2021 16:11:44 GMT (338kb,D)
[v2] Mon, 15 Nov 2021 14:37:49 GMT (340kb,D)

Link back to: arXiv, form interface, contact.