Current browse context:
math.ST
Change to browse by:
References & Citations
Mathematics > Statistics Theory
Title: A Unified Framework for High-Dimensional Analysis of M-Estimators with Decomposable Regularizers
(Submitted on 13 Oct 2010 (v1), last revised 12 Mar 2013 (this version, v3))
Abstract: High-dimensional statistical inference deals with models in which the the number of parameters p is comparable to or larger than the sample size n. Since it is usually impossible to obtain consistent procedures unless $p/n\rightarrow0$, a line of recent work has studied models with various types of low-dimensional structure, including sparse vectors, sparse and structured matrices, low-rank matrices and combinations thereof. In such settings, a general approach to estimation is to solve a regularized optimization problem, which combines a loss function measuring how well the model fits the data with some regularization function that encourages the assumed structure. This paper provides a unified framework for establishing consistency and convergence rates for such regularized M-estimators under high-dimensional scaling. We state one main theorem and show how it can be used to re-derive some existing results, and also to obtain a number of new results on consistency and convergence rates, in both $\ell_2$-error and related norms. Our analysis also identifies two key properties of loss and regularization functions, referred to as restricted strong convexity and decomposability, that ensure corresponding regularized M-estimators have fast convergence rates and which are optimal in many well-studied cases.
Submission history
From: Sahand N. Negahban [view email][v1] Wed, 13 Oct 2010 19:05:27 GMT (98kb)
[v2] Wed, 14 Mar 2012 14:28:14 GMT (78kb)
[v3] Tue, 12 Mar 2013 11:17:04 GMT (157kb)
Link back to: arXiv, form interface, contact.