Current browse context:
stat.ME
Change to browse by:
References & Citations
Statistics > Methodology
Title: Surrogate Aided Unsupervised Recovery of Sparse Signals in Single Index Models for Binary Outcomes
(Submitted on 18 Jan 2017 (v1), last revised 1 Jul 2018 (this version, v3))
Abstract: We consider the recovery of regression coefficients, denoted by $\boldsymbol{\beta}_0$, for a single index model (SIM) relating a binary outcome $Y$ to a set of possibly high dimensional covariates $\boldsymbol{X}$, based on a large but 'unlabeled' dataset $\mathcal{U}$, with $Y$ never observed. On $\mathcal{U}$, we fully observe $\boldsymbol{X}$ and additionally, a surrogate $S$ which, while not being strongly predictive of $Y$ throughout the entirety of its support, can forecast it with high accuracy when it assumes extreme values. Such datasets arise naturally in modern studies involving large databases such as electronic medical records (EMR) where $Y$, unlike $(\boldsymbol{X}, S)$, is difficult and/or expensive to obtain. In EMR studies, an example of $Y$ and $S$ would be the true disease phenotype and the count of the associated diagnostic codes respectively. Assuming another SIM for $S$ given $\boldsymbol{X}$, we show that under sparsity assumptions, we can recover $\boldsymbol{\beta}_0$ proportionally by simply fitting a least squares LASSO estimator to the subset of the observed data on $(\boldsymbol{X}, S)$ restricted to the extreme sets of $S$, with $Y$ imputed using the surrogacy of $S$. We obtain sharp finite sample performance bounds for our estimator, including deterministic deviation bounds and probabilistic guarantees. We demonstrate the effectiveness of our approach through multiple simulation studies, as well as by application to real data from an EMR study conducted at the Partners HealthCare Systems.
Submission history
From: Abhishek Chakrabortty [view email][v1] Wed, 18 Jan 2017 20:51:27 GMT (93kb,D)
[v2] Sat, 19 Aug 2017 17:59:30 GMT (125kb,D)
[v3] Sun, 1 Jul 2018 01:09:51 GMT (82kb,D)
Link back to: arXiv, form interface, contact.