References & Citations
Mathematics > Statistics Theory
Title: Minimax risks for sparse regressions: Ultra-high-dimensional phenomenons
(Submitted on 3 Aug 2010 (v1), last revised 24 Jan 2012 (this version, v3))
Abstract: Consider the standard Gaussian linear regression model $Y=X\theta+\epsilon$, where $Y\in R^n$ is a response vector and $ X\in R^{n*p}$ is a design matrix. Numerous work have been devoted to building efficient estimators of $\theta$ when $p$ is much larger than $n$. In such a situation, a classical approach amounts to assume that $\theta_0$ is approximately sparse. This paper studies the minimax risks of estimation and testing over classes of $k$-sparse vectors $\theta$. These bounds shed light on the limitations due to high-dimensionality. The results encompass the problem of prediction (estimation of $X\theta$), the inverse problem (estimation of $\theta_0$) and linear testing (testing $X\theta=0$). Interestingly, an elbow effect occurs when the number of variables $k\log(p/k)$ becomes large compared to $n$. Indeed, the minimax risks and hypothesis separation distances blow up in this ultra-high dimensional setting. We also prove that even dimension reduction techniques cannot provide satisfying results in an ultra-high dimensional setting. Moreover, we compute the minimax risks when the variance of the noise is unknown. The knowledge of this variance is shown to play a significant role in the optimal rates of estimation and testing. All these minimax bounds provide a characterization of statistical problems that are so difficult so that no procedure can provide satisfying results.
Submission history
From: Nicolas Verzelen [view email][v1] Tue, 3 Aug 2010 12:04:05 GMT (74kb)
[v2] Mon, 20 Sep 2010 07:42:56 GMT (80kb)
[v3] Tue, 24 Jan 2012 13:08:58 GMT (173kb)
Link back to: arXiv, form interface, contact.