Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Cross-validation Confidence Intervals for Test Error
(Submitted on 24 Jul 2020 (v1), last revised 31 Oct 2020 (this version, v2))
Abstract: This work develops central limit theorems for cross-validation and consistent estimators of its asymptotic variance under weak stability conditions on the learning algorithm. Together, these results provide practical, asymptotically-exact confidence intervals for $k$-fold test error and valid, powerful hypothesis tests of whether one learning algorithm has smaller $k$-fold test error than another. These results are also the first of their kind for the popular choice of leave-one-out cross-validation. In our real-data experiments with diverse learning algorithms, the resulting intervals and tests outperform the most popular alternative methods from the literature.
Submission history
From: Pierre Bayle [view email][v1] Fri, 24 Jul 2020 17:40:06 GMT (262kb,D)
[v2] Sat, 31 Oct 2020 17:24:26 GMT (278kb,D)
Link back to: arXiv, form interface, contact.