Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: The Chi-Square Test of Distance Correlation
(Submitted on 27 Dec 2019 (v1), revised 7 Jan 2020 (this version, v2), latest version 14 May 2021 (v5))
Abstract: Distance correlation has gained much recent attention in the statistics and machine learning community: the sample statistic is straightforward to compute, works for any metric or kernel choice, and asymptotically equals zero if and only if independence. One major bottleneck is the testing process: the null distribution of distance correlation depends on the metric choice and marginal distributions, which cannot be easily estimated. To compute a p-value, the standard approach is to estimate the null distribution via permutation, which is very costly for large amount of data. In this paper, we propose a chi-square distribution to approximate the null distribution of the unbiased distance correlation. We prove that the chi-square distribution either equals or well-approximates the null distribution, and always upper tail dominates the null distribution. The resulting distance correlation chi-square test does not require any permutation nor parameter estimation, works with any strong negative type metric or characteristic kernel, is valid and universally consistent for testing independence, and enjoys a similar finite-sample testing power as the standard permutation test. For one-dimensional data using Euclidean distance, testing independence using distance correlation now runs in linear time complexity, rendering it comparable in speed to the Pearson correlation t-test. The results are supported and demonstrated via simulations.
Submission history
From: Cencheng Shen [view email][v1] Fri, 27 Dec 2019 15:16:40 GMT (144kb,D)
[v2] Tue, 7 Jan 2020 15:08:41 GMT (142kb,D)
[v3] Wed, 22 Jan 2020 21:53:47 GMT (143kb,D)
[v4] Fri, 21 Feb 2020 21:35:39 GMT (143kb,D)
[v5] Fri, 14 May 2021 18:09:51 GMT (154kb,D)
Link back to: arXiv, form interface, contact.