We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.ST

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Statistics Theory

Title: On cross-validated Lasso in high dimensions

Abstract: In this paper, we derive non-asymptotic error bounds for the Lasso estimator when the penalty parameter for the estimator is chosen using $K$-fold cross-validation. Our bounds imply that the cross-validated Lasso estimator has nearly optimal rates of convergence in the prediction, $L^2$, and $L^1$ norms. For example, we show that in the model with the Gaussian noise and under fairly general assumptions on the candidate set of values of the penalty parameter, the estimation error of the cross-validated Lasso estimator converges to zero in the prediction norm with the $\sqrt{s\log p / n}\times \sqrt{\log(p n)}$ rate, where $n$ is the sample size of available data, $p$ is the number of covariates, and $s$ is the number of non-zero coefficients in the model. Thus, the cross-validated Lasso estimator achieves the fastest possible rate of convergence in the prediction norm up to a small logarithmic factor $\sqrt{\log(p n)}$, and similar conclusions apply for the convergence rate both in $L^2$ and in $L^1$ norms. Importantly, our results cover the case when $p$ is (potentially much) larger than $n$ and also allow for the case of non-Gaussian noise. Our paper therefore serves as a justification for the widely spread practice of using cross-validation as a method to choose the penalty parameter for the Lasso estimator.
Subjects: Statistics Theory (math.ST)
Cite as: arXiv:1605.02214 [math.ST]
  (or arXiv:1605.02214v6 [math.ST] for this version)

Submission history

From: Denis Chetverikov [view email]
[v1] Sat, 7 May 2016 16:52:32 GMT (25kb)
[v2] Fri, 2 Sep 2016 17:15:48 GMT (116kb,D)
[v3] Sat, 8 Jul 2017 10:30:30 GMT (116kb,D)
[v4] Wed, 30 Jan 2019 17:25:24 GMT (118kb,D)
[v5] Tue, 13 Aug 2019 07:29:48 GMT (118kb,D)
[v6] Thu, 6 Feb 2020 18:17:55 GMT (98kb)

Link back to: arXiv, form interface, contact.