References & Citations
Mathematics > Statistics Theory
Title: On concentration for (regularized) empirical risk minimization
(Submitted on 2 Dec 2015 (v1), last revised 11 Jan 2016 (this version, v2))
Abstract: Rates of convergence for empirical risk minimizers have been well studied in the literature. In this paper, we aim to provide a complementary set of results, in particular by showing that after normalization, the risk of the empirical minimizer concentrates on a single point. Such results have been established by~\cite{chatterjee2014new} for constrained estimators in the normal sequence model. We first generalize and sharpen this result to regularized least squares with convex penalties, making use of a "direct" argument based on Borell's theorem. We then study generalizations to other loss functions, including the negative log-likelihood for exponential families combined with a strictly convex regularization penalty. The results in this general setting are based on more "indirect" arguments as well as on concentration inequalities for maxima of empirical processes.
Submission history
From: Sara van de Geer [view email][v1] Wed, 2 Dec 2015 12:54:09 GMT (16kb)
[v2] Mon, 11 Jan 2016 11:16:58 GMT (23kb)
Link back to: arXiv, form interface, contact.