We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.DS

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Computer Science > Data Structures and Algorithms

Title: Testing Ising Models

Abstract: Given samples from an unknown multivariate distribution $p$, is it possible to distinguish whether $p$ is the product of its marginals versus $p$ being far from every product distribution? Similarly, is it possible to distinguish whether $p$ equals a given distribution $q$ versus $p$ and $q$ being far from each other? These problems of testing independence and goodness-of-fit have received enormous attention in statistics, information theory, and theoretical computer science, with sample-optimal algorithms known in several interesting regimes of parameters. Unfortunately, it has also been understood that these problems become intractable in large dimensions, necessitating exponential sample complexity.
Motivated by the exponential lower bounds for general distributions as well as the ubiquity of Markov Random Fields (MRFs) in the modeling of high-dimensional distributions, we initiate the study of distribution testing on structured multivariate distributions, and in particular the prototypical example of MRFs: the Ising Model. We demonstrate that, in this structured setting, we can avoid the curse of dimensionality, obtaining sample and time efficient testers for independence and goodness-of-fit. One of the key technical challenges we face along the way is bounding the variance of functions of the Ising model.
Comments: Appeared SODA 2018. Final version to appear in IEEE Transactions on Information Theory
Subjects: Data Structures and Algorithms (cs.DS); Information Theory (cs.IT); Machine Learning (cs.LG); Probability (math.PR); Statistics Theory (math.ST)
Cite as: arXiv:1612.03147 [cs.DS]
  (or arXiv:1612.03147v6 [cs.DS] for this version)

Submission history

From: Gautam Kamath [view email]
[v1] Fri, 9 Dec 2016 20:04:56 GMT (334kb,D)
[v2] Thu, 15 Dec 2016 05:34:14 GMT (323kb,D)
[v3] Fri, 7 Apr 2017 15:06:28 GMT (328kb,D)
[v4] Mon, 30 Oct 2017 20:34:46 GMT (313kb,D)
[v5] Tue, 5 Feb 2019 16:33:20 GMT (72kb)
[v6] Wed, 10 Jul 2019 22:02:20 GMT (65kb)

Link back to: arXiv, form interface, contact.