We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.IT

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Computer Science > Information Theory

Title: Multivariate Dependence Beyond Shannon Information

Abstract: Accurately determining dependency structure is critical to discovering a system's causal organization. We recently showed that the transfer entropy fails in a key aspect of this---measuring information flow---due to its conflation of dyadic and polyadic relationships. We extend this observation to demonstrate that this is true of all such Shannon information measures when used to analyze multivariate dependencies. This has broad implications, particularly when employing information to express the organization and mechanisms embedded in complex systems, including the burgeoning efforts to combine complex network theory with information theory. Here, we do not suggest that any aspect of information theory is wrong. Rather, the vast majority of its informational measures are simply inadequate for determining the meaningful dependency structure within joint probability distributions. Therefore, such information measures are inadequate for discovering intrinsic causal relations. We close by demonstrating that such distributions exist across an arbitrary set of variables.
Comments: 10 pages, 6 figures, 3 tables; this http URL
Subjects: Information Theory (cs.IT); Statistical Mechanics (cond-mat.stat-mech); Statistics Theory (math.ST); Machine Learning (stat.ML)
DOI: 10.3390/e19100531
Cite as: arXiv:1609.01233 [cs.IT]
  (or arXiv:1609.01233v2 [cs.IT] for this version)

Submission history

From: James P. Crutchfield [view email]
[v1] Mon, 5 Sep 2016 17:38:07 GMT (373kb,D)
[v2] Thu, 8 Sep 2016 23:07:39 GMT (369kb,D)

Link back to: arXiv, form interface, contact.