We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: AI-SARAH: Adaptive and Implicit Stochastic Recursive Gradient Methods

Abstract: We present AI-SARAH, a practical variant of SARAH. As a variant of SARAH, this algorithm employs the stochastic recursive gradient yet adjusts step-size based on local geometry. AI-SARAH implicitly computes step-size and efficiently estimates local Lipschitz smoothness of stochastic functions. It is fully adaptive, tune-free, straightforward to implement, and computationally efficient. We provide technical insight and intuitive illustrations on its design and convergence. We conduct extensive empirical analysis and demonstrate its strong performance compared with its classical counterparts and other state-of-the-art first-order methods in solving convex machine learning problems.
Subjects: Machine Learning (cs.LG); Optimization and Control (math.OC)
Cite as: arXiv:2102.09700 [cs.LG]
  (or arXiv:2102.09700v3 [cs.LG] for this version)

Submission history

From: Martin Takáč [view email]
[v1] Fri, 19 Feb 2021 01:17:15 GMT (3762kb,D)
[v2] Mon, 31 Jan 2022 17:55:55 GMT (20146kb,D)
[v3] Tue, 1 Feb 2022 17:24:49 GMT (20146kb,D)

Link back to: arXiv, form interface, contact.