We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.ST

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Mathematics > Statistics Theory

Title: The degrees of freedom of the Lasso for general design matrix

Abstract: In this paper, we investigate the degrees of freedom ($\dof$) of penalized $\ell_1$ minimization (also known as the Lasso) for linear regression models. We give a closed-form expression of the $\dof$ of the Lasso response. Namely, we show that for any given Lasso regularization parameter $\lambda$ and any observed data $y$ belonging to a set of full (Lebesgue) measure, the cardinality of the support of a particular solution of the Lasso problem is an unbiased estimator of the degrees of freedom. This is achieved without the need of uniqueness of the Lasso solution. Thus, our result holds true for both the underdetermined and the overdetermined case, where the latter was originally studied in \cite{zou}. We also show, by providing a simple counterexample, that although the $\dof$ theorem of \cite{zou} is correct, their proof contains a flaw since their divergence formula holds on a different set of a full measure than the one that they claim. An effective estimator of the number of degrees of freedom may have several applications including an objectively guided choice of the regularization parameter in the Lasso through the $\sure$ framework. Our theoretical findings are illustrated through several numerical simulations.
Comments: A short version appeared in SPARS'11, June 2011 Previously entitled "The degrees of freedom of penalized l1 minimization"
Subjects: Statistics Theory (math.ST); Information Theory (cs.IT)
Cite as: arXiv:1111.1162 [math.ST]
  (or arXiv:1111.1162v3 [math.ST] for this version)

Submission history

From: Jalal Fadili [view email]
[v1] Fri, 4 Nov 2011 15:42:36 GMT (321kb,D)
[v2] Tue, 27 Mar 2012 09:30:09 GMT (155kb,D)
[v3] Mon, 28 May 2012 20:35:58 GMT (4158kb,D)

Link back to: arXiv, form interface, contact.