References & Citations
Mathematics > Statistics Theory
Title: On cross-validated Lasso
(Submitted on 7 May 2016 (v1), revised 8 Jul 2017 (this version, v3), latest version 6 Feb 2020 (v6))
Abstract: In this paper, we derive a rate of convergence of the Lasso estimator when the penalty parameter $\lambda$ for the estimator is chosen using $K$-fold cross-validation; in particular, we show that in the model with the Gaussian noise and under fairly general assumptions on the candidate set of values of $\lambda$, the prediction norm of the estimation error of the cross-validated Lasso estimator is with high probability bounded from above up to a constant by $(s\log p /n)^{1/2}\cdot \log^{7/8}(p n)$, where $n$ is the sample size of available data, $p$ is the number of covariates, and $s$ is the number of non-zero coefficients in the model. Thus, the cross-validated Lasso estimator achieves the fastest possible rate of convergence up to a small logarithmic factor $\log^{7/8}(p n)$. In addition, we derive a sparsity bound for the cross-validated Lasso estimator; in particular, we show that under the same conditions as above, the number of non-zero coefficients of the estimator is with high probability bounded from above up to a constant by $s\log^5(p n)$. Finally, we show that our proof technique generates non-trivial bounds on the prediction norm of the estimation error of the cross-validated Lasso estimator even if the assumption of the Gaussian noise fails; in particular, the prediction norm of the estimation error is with high-probability bounded from above up to a constant by $(s\log^2(p n)/n)^{1/4}$ under mild regularity conditions.
Submission history
From: Denis Chetverikov [view email][v1] Sat, 7 May 2016 16:52:32 GMT (25kb)
[v2] Fri, 2 Sep 2016 17:15:48 GMT (116kb,D)
[v3] Sat, 8 Jul 2017 10:30:30 GMT (116kb,D)
[v4] Wed, 30 Jan 2019 17:25:24 GMT (118kb,D)
[v5] Tue, 13 Aug 2019 07:29:48 GMT (118kb,D)
[v6] Thu, 6 Feb 2020 18:17:55 GMT (98kb)
Link back to: arXiv, form interface, contact.