We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Efficient learning with robust gradient descent

Abstract: Minimizing the empirical risk is a popular training strategy, but for learning tasks where the data may be noisy or heavy-tailed, one may require many observations in order to generalize well. To achieve better performance under less stringent requirements, we introduce a procedure which constructs a robust approximation of the risk gradient for use in an iterative learning routine. Using high-probability bounds on the excess risk of this algorithm, we show that our update does not deviate far from the ideal gradient-based update. Empirical tests using both controlled simulations and real-world benchmark data show that in diverse settings, the proposed procedure can learn more efficiently, using less resources (iterations and observations) while generalizing better.
Subjects: Machine Learning (stat.ML)
Cite as: arXiv:1706.00182 [stat.ML]
  (or arXiv:1706.00182v3 [stat.ML] for this version)

Submission history

From: Matthew J. Holland [view email]
[v1] Thu, 1 Jun 2017 07:00:56 GMT (1126kb,D)
[v2] Wed, 10 Oct 2018 09:32:31 GMT (452kb,D)
[v3] Mon, 15 Oct 2018 02:51:50 GMT (452kb,D)

Link back to: arXiv, form interface, contact.