Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: A SMART Stochastic Algorithm for Nonconvex Optimization with Applications to Robust Machine Learning
(Submitted on 4 Oct 2016 (v1), last revised 5 Feb 2017 (this version, v2))
Abstract: In this paper, we show how to transform any optimization problem that arises from fitting a machine learning model into one that (1) detects and removes contaminated data from the training set while (2) simultaneously fitting the trimmed model on the uncontaminated data that remains. To solve the resulting nonconvex optimization problem, we introduce a fast stochastic proximal-gradient algorithm that incorporates prior knowledge through nonsmooth regularization. For datasets of size $n$, our approach requires $O(n^{2/3}/\varepsilon)$ gradient evaluations to reach $\varepsilon$-accuracy and, when a certain error bound holds, the complexity improves to $O(\kappa n^{2/3}\log(1/\varepsilon))$. These rates are $n^{1/3}$ times better than those achieved by typical, full gradient methods.
Submission history
From: Damek Davis [view email][v1] Tue, 4 Oct 2016 17:24:43 GMT (2895kb,D)
[v2] Sun, 5 Feb 2017 15:24:39 GMT (1221kb,D)
Link back to: arXiv, form interface, contact.