We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: A Stochastic Trust Region Algorithm Based on Careful Step Normalization

Abstract: An algorithm is proposed for solving stochastic and finite sum minimization problems. Based on a trust region methodology, the algorithm employs normalized steps, at least as long as the norms of the stochastic gradient estimates are within a specified interval. The complete algorithm---which dynamically chooses whether or not to employ normalized steps---is proved to have convergence guarantees that are similar to those possessed by a traditional stochastic gradient approach under various sets of conditions related to the accuracy of the stochastic gradient estimates and choice of stepsize sequence. The results of numerical experiments are presented when the method is employed to minimize convex and nonconvex machine learning test problems. These results illustrate that the method can outperform a traditional stochastic gradient approach.
Subjects: Optimization and Control (math.OC); Machine Learning (stat.ML)
Report number: Lehigh ISE/COR@L Technical Report 17T-017
Cite as: arXiv:1712.10277 [math.OC]
  (or arXiv:1712.10277v3 [math.OC] for this version)

Submission history

From: Frank E. Curtis [view email]
[v1] Fri, 29 Dec 2017 16:59:25 GMT (447kb,D)
[v2] Sat, 27 Jan 2018 16:19:08 GMT (447kb,D)
[v3] Tue, 26 Jun 2018 16:14:56 GMT (624kb,D)

Link back to: arXiv, form interface, contact.