References & Citations
Mathematics > Statistics Theory
Title: Remember the Curse of Dimensionality: The Case of Goodness-of-Fit Testing in Arbitrary Dimension
(Submitted on 27 Jul 2016 (v1), last revised 12 Sep 2018 (this version, v3))
Abstract: Despite a substantial literature on nonparametric two-sample goodness-of-fit testing in arbitrary dimensions spanning decades, there is no mention there of any curse of dimensionality. Only more recently Ramdas et al. (2015) have discussed this issue in the context of kernel methods by showing that their performance degrades with the dimension even when the underlying distributions are isotropic Gaussians. We take a minimax perspective and follow in the footsteps of Ingster (1987) to derive the minimax rate in arbitrary dimension when the discrepancy is measured in the L2 metric. That rate is revealed to be nonparametric and exhibit a prototypical curse of dimensionality. We further extend Ingster's work to show that the chi-squared test achieves the minimax rate. Moreover, we show that the test can be made to work when the distributions have support of low intrinsic dimension. Finally, inspired by Ingster (2000), we consider a multiscale version of the chi-square test which can adapt to unknown smoothness and/or unknown intrinsic dimensionality without much loss in power.
Submission history
From: Ery Arias-Castro [view email][v1] Wed, 27 Jul 2016 15:41:10 GMT (23kb)
[v2] Tue, 11 Jul 2017 18:55:47 GMT (24kb)
[v3] Wed, 12 Sep 2018 01:23:44 GMT (24kb)
Link back to: arXiv, form interface, contact.