Current browse context:
math.ST
Change to browse by:
References & Citations
Mathematics > Statistics Theory
Title: The All-or-Nothing Phenomenon in Sparse Tensor PCA
(Submitted on 22 Jul 2020 (v1), last revised 5 Sep 2020 (this version, v2))
Abstract: We study the statistical problem of estimating a rank-one sparse tensor corrupted by additive Gaussian noise, a model also known as sparse tensor PCA. We show that for Bernoulli and Bernoulli-Rademacher distributed signals and \emph{for all} sparsity levels which are sublinear in the dimension of the signal, the sparse tensor PCA model exhibits a phase transition called the \emph{all-or-nothing phenomenon}. This is the property that for some signal-to-noise ratio (SNR) $\mathrm{SNR_c}$ and any fixed $\epsilon>0$, if the SNR of the model is below $\left(1-\epsilon\right)\mathrm{SNR_c}$, then it is impossible to achieve any arbitrarily small constant correlation with the hidden signal, while if the SNR is above $\left(1+\epsilon \right)\mathrm{SNR_c}$, then it is possible to achieve almost perfect correlation with the hidden signal. The all-or-nothing phenomenon was initially established in the context of sparse linear regression, and over the last year also in the context of sparse 2-tensor (matrix) PCA, Bernoulli group testing, and generalized linear models. Our results follow from a more general result showing that for any Gaussian additive model with a discrete uniform prior, the all-or-nothing phenomenon follows as a direct outcome of an appropriately defined "near-orthogonality" property of the support of the prior distribution.
Submission history
From: Ilias Zadik [view email][v1] Wed, 22 Jul 2020 00:02:48 GMT (21kb)
[v2] Sat, 5 Sep 2020 14:52:00 GMT (21kb)
Link back to: arXiv, form interface, contact.