Current browse context:
math.OC
Change to browse by:
References & Citations
Mathematics > Optimization and Control
Title: LSOS: Line-search Second-Order Stochastic optimization methods for nonconvex finite sums
(Submitted on 31 Jul 2020 (v1), last revised 27 Jun 2022 (this version, v3))
Abstract: We develop a line-search second-order algorithmic framework for minimizing finite sums. We do not make any convexity assumptions, but require the terms of the sum to be continuously differentiable and have Lipschitz-continuous gradients. The methods fitting into this framework combine line searches and suitably decaying step lengths. A key issue is a two-step sampling at each iteration, which allows us to control the error present in the line-search procedure. Stationarity of limit points is proved in the almost-sure sense, while almost-sure convergence of the sequence of approximations to the solution holds with the additional hypothesis that the functions are strongly convex. Numerical experiments, including comparisons with state-of-the art stochastic optimization methods, show the efficiency of our approach.
Submission history
From: Marco Viola [view email][v1] Fri, 31 Jul 2020 11:07:40 GMT (776kb)
[v2] Thu, 10 Jun 2021 14:55:46 GMT (260kb)
[v3] Mon, 27 Jun 2022 09:28:49 GMT (401kb)
Link back to: arXiv, form interface, contact.