We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.ST

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Mathematics > Statistics Theory

Title: BET on Independence

Authors: Kai Zhang
Abstract: We study the problem of nonparametric dependence detection. Many existing methods may suffer severe power loss due to non-uniform consistency, which we illustrate with a paradox. To avoid such power loss, we approach the nonparametric test of independence through the new framework of binary expansion statistics (BEStat) and binary expansion testing (BET), which examine dependence through a novel binary expansion filtration approximation of the copula. Through a Hadamard transform, we find that the symmetry statistics in the filtration are complete sufficient statistics for dependence. These statistics are also uncorrelated under the null. By utilizing symmetry statistics, the BET avoids the problem of non-uniform consistency and improves upon a wide class of commonly used methods (a) by achieving the minimax rate in sample size requirement for reliable power and (b) by providing clear interpretations of global relationships upon rejection of independence. The binary expansion approach also connects the symmetry statistics with the current computing system to facilitate efficient bitwise implementation. We illustrate the BET with a study of the distribution of stars in the night sky and with an exploratory data analysis of the TCGA breast cancer data.
Subjects: Statistics Theory (math.ST); Machine Learning (cs.LG); Computation (stat.CO); Methodology (stat.ME); Machine Learning (stat.ML)
DOI: 10.1080/01621459.2018.1537921
Cite as: arXiv:1610.05246 [math.ST]
  (or arXiv:1610.05246v7 [math.ST] for this version)

Submission history

From: Kai Zhang [view email]
[v1] Mon, 17 Oct 2016 18:19:49 GMT (69kb,D)
[v2] Thu, 12 Jan 2017 03:26:00 GMT (80kb,D)
[v3] Thu, 26 Jan 2017 07:09:37 GMT (81kb,D)
[v4] Sun, 23 Apr 2017 02:08:08 GMT (116kb,D)
[v5] Mon, 20 Nov 2017 15:57:14 GMT (141kb,D)
[v6] Sun, 13 May 2018 02:25:46 GMT (133kb,D)
[v7] Mon, 15 Apr 2019 20:39:38 GMT (133kb,D)

Link back to: arXiv, form interface, contact.