We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.NA

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Numerical Analysis

Title: Quadratic and Cubic Regularisation Methods with Inexact function and Random Derivatives for Finite-Sum Minimisation

Abstract: This paper focuses on regularisation methods using models up to the third order to search for up to second-order critical points of a finite-sum minimisation problem. The variant presented belongs to the framework of [3]: it employs random models with accuracy guaranteed with a sufficiently large prefixed probability and deterministic inexact function evaluations within a prescribed level of accuracy. Without assuming unbiased estimators, the expected number of iterations is $\mathcal{O}\bigl(\epsilon_1^{-2}\bigr)$ or $\mathcal{O}\bigl(\epsilon_1^{-{3/2}}\bigr)$ when searching for a first-order critical point using a second or third order model, respectively, and of $\mathcal{O}\bigl(\max[\epsilon_1^{-{3/2}},\epsilon_2^{-3}]\bigr)$ when seeking for second-order critical points with a third order model, in which $\epsilon_j$, $j\in\{1,2\}$, is the $j$th-order tolerance. These results match the worst-case optimal complexity for the deterministic counterpart of the method. Preliminary numerical tests for first-order optimality in the context of nonconvex binary classification in imaging, with and without Artifical Neural Networks (ANNs), are presented and discussed.
Comments: 9 pages
Subjects: Numerical Analysis (math.NA)
Cite as: arXiv:2104.00592 [math.NA]
  (or arXiv:2104.00592v2 [math.NA] for this version)

Submission history

From: Gianmarco Gurioli [view email]
[v1] Tue, 30 Mar 2021 15:49:50 GMT (100kb)
[v2] Fri, 2 Apr 2021 08:12:12 GMT (66kb)

Link back to: arXiv, form interface, contact.