We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ME

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Methodology

Title: Estimation of prediction error with known covariate shift

Abstract: In supervised learning, the estimation of prediction error on unlabeled test data is an important task. Existing methods are usually built on the assumption that the training and test data are sampled from the same distribution, which is often violated in practice. As a result, traditional estimators like cross-validation (CV) will be biased and this may result in poor model selection. In this paper, we assume that we have a test dataset in which the feature values are available but not the outcome labels, and focus on a particular form of distributional shift called "covariate shift". We propose an alternative method based on parametric bootstrap of the target of conditional error. Empirically, our method outperforms CV for both simulation and real data example across different modeling tasks.
Subjects: Methodology (stat.ME)
Cite as: arXiv:2205.01849 [stat.ME]
  (or arXiv:2205.01849v2 [stat.ME] for this version)

Submission history

From: Hui Xu [view email]
[v1] Wed, 4 May 2022 02:07:13 GMT (353kb,D)
[v2] Thu, 29 Sep 2022 00:49:32 GMT (1257kb,D)

Link back to: arXiv, form interface, contact.