We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Adaptive Sampling Quasi-Newton Methods for Zeroth-Order Stochastic Optimization

Abstract: We consider unconstrained stochastic optimization problems with no available gradient information. Such problems arise in settings from derivative-free simulation optimization to reinforcement learning. We propose an adaptive sampling quasi-Newton method where we estimate the gradients of a stochastic function using finite differences within a common random number framework. We develop modified versions of a norm test and an inner product quasi-Newton test to control the sample sizes used in the stochastic approximations and provide global convergence results to the neighborhood of the optimal solution. We present numerical experiments on simulation optimization problems to illustrate the performance of the proposed algorithm. When compared with classical zeroth-order stochastic gradient methods, we observe that our strategies of adapting the sample sizes significantly improve performance in terms of the number of stochastic function evaluations required.
Subjects: Optimization and Control (math.OC); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
Cite as: arXiv:2109.12213 [math.OC]
  (or arXiv:2109.12213v1 [math.OC] for this version)

Submission history

From: Stefan Wild [view email]
[v1] Fri, 24 Sep 2021 21:49:25 GMT (1380kb,D)

Link back to: arXiv, form interface, contact.