We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.ST

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Mathematics > Statistics Theory

Title: Remember the Curse of Dimensionality: The Case of Goodness-of-Fit Testing in Arbitrary Dimension

Abstract: Despite a substantial literature on nonparametric two-sample goodness-of-fit testing in arbitrary dimensions spanning decades, there is no mention there of any curse of dimensionality. Only more recently Ramdas et al. (2015) have discussed this issue in the context of kernel methods by showing that their performance degrades with the dimension even when the underlying distributions are isotropic Gaussians. We take a minimax perspective and follow in the footsteps of Ingster (1987) to derive the minimax rate in arbitrary dimension when the discrepancy is measured in the L2 metric. That rate is revealed to be nonparametric and exhibit a prototypical curse of dimensionality. We further extend Ingster's work to show that the chi-squared test achieves the minimax rate. Moreover, we show that the test can be made to work when the distributions have support of low intrinsic dimension. Finally, inspired by Ingster (2000), we consider a multiscale version of the chi-square test which can adapt to unknown smoothness and/or unknown intrinsic dimensionality without much loss in power.
Comments: This version comes after the publication of the paper in the Journal of Nonparametric Statistics. The main change is to cite the work of Ramdas et al. Some very minor typos were also corrected
Subjects: Statistics Theory (math.ST)
Cite as: arXiv:1607.08156 [math.ST]
  (or arXiv:1607.08156v3 [math.ST] for this version)

Submission history

From: Ery Arias-Castro [view email]
[v1] Wed, 27 Jul 2016 15:41:10 GMT (23kb)
[v2] Tue, 11 Jul 2017 18:55:47 GMT (24kb)
[v3] Wed, 12 Sep 2018 01:23:44 GMT (24kb)

Link back to: arXiv, form interface, contact.