We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Adaptive Zeroth-Order Optimisation of Nonconvex Composite Objectives

Abstract: In this paper, we propose and analyze algorithms for zeroth-order optimization of non-convex composite objectives, focusing on reducing the complexity dependence on dimensionality. This is achieved by exploiting the low dimensional structure of the decision set using the stochastic mirror descent method with an entropy alike function, which performs gradient descent in the space equipped with the maximum norm. To improve the gradient estimation, we replace the classic Gaussian smoothing method with a sampling method based on the Rademacher distribution and show that the mini-batch method copes with the non-Euclidean geometry. To avoid tuning hyperparameters, we analyze the adaptive stepsizes for the general stochastic mirror descent and show that the adaptive version of the proposed algorithm converges without requiring prior knowledge about the problem.
Subjects: Optimization and Control (math.OC); Machine Learning (cs.LG)
Cite as: arXiv:2208.04579 [math.OC]
  (or arXiv:2208.04579v2 [math.OC] for this version)

Submission history

From: Weijia Shao [view email]
[v1] Tue, 9 Aug 2022 07:36:25 GMT (344kb,D)
[v2] Sun, 14 Aug 2022 21:49:14 GMT (344kb,D)

Link back to: arXiv, form interface, contact.