We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Adaptive Newton Method for Empirical Risk Minimization to Statistical Accuracy

Abstract: We consider empirical risk minimization for large-scale datasets. We introduce Ada Newton as an adaptive algorithm that uses Newton's method with adaptive sample sizes. The main idea of Ada Newton is to increase the size of the training set by a factor larger than one in a way that the minimization variable for the current training set is in the local neighborhood of the optimal argument of the next training set. This allows to exploit the quadratic convergence property of Newton's method and reach the statistical accuracy of each training set with only one iteration of Newton's method. We show theoretically and empirically that Ada Newton can double the size of the training set in each iteration to achieve the statistical accuracy of the full training set with about two passes over the dataset.
Subjects: Machine Learning (cs.LG); Optimization and Control (math.OC)
Cite as: arXiv:1605.07659 [cs.LG]
  (or arXiv:1605.07659v1 [cs.LG] for this version)

Submission history

From: Aryan Mokhtari [view email]
[v1] Tue, 24 May 2016 21:02:50 GMT (50kb,D)

Link back to: arXiv, form interface, contact.