We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ME

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Methodology

Title: Asymptotic equivalence of regularization methods in thresholded parameter space

Abstract: High-dimensional data analysis has motivated a spectrum of regularization methods for variable selection and sparse modeling, with two popular classes of convex ones and concave ones. A long debate has been on whether one class dominates the other, an important question both in theory and to practitioners. In this paper, we characterize the asymptotic equivalence of regularization methods, with general penalty functions, in a thresholded parameter space under the generalized linear model setting, where the dimensionality can grow up to exponentially with the sample size. To assess their performance, we establish the oracle inequalities, as in Bickel, Ritov and Tsybakov (2009), of the global minimizer for these methods under various prediction and variable selection losses. These results reveal an interesting phase transition phenomenon. For polynomially growing dimensionality, the $L_1$-regularization method of Lasso and concave methods are asymptotically equivalent, having the same convergence rates in the oracle inequalities. For exponentially growing dimensionality, concave methods are asymptotically equivalent but have faster convergence rates than the Lasso. We also establish a stronger property of the oracle risk inequalities of the regularization methods, as well as the sampling properties of computable solutions. Our new theoretical results are illustrated and justified by simulation and real data examples.
Comments: 39 pages, 3 figures
Subjects: Methodology (stat.ME); Statistics Theory (math.ST); Machine Learning (stat.ML)
MSC classes: 62J07(Primary) 62F07(Secondary)
Journal reference: Journal of the American Statistical Association 108, 1044-1061
DOI: 10.1080/01621459.2013.803972
Cite as: arXiv:1605.03310 [stat.ME]
  (or arXiv:1605.03310v1 [stat.ME] for this version)

Submission history

From: Jinchi Lv [view email]
[v1] Wed, 11 May 2016 07:39:16 GMT (45kb)

Link back to: arXiv, form interface, contact.