We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Physics > Data Analysis, Statistics and Probability

Title: Inferring the shape of data: A probabilistic framework for analyzing experiments in the natural sciences

Abstract: A critical step in data analysis for many different types of experiments is the identification of features with theoretically defined shapes in N-dimensional datasets; examples of this process include finding peaks in multi-dimensional molecular spectra or emitters in fluorescence microscopy images. Identifying such features involves determining if the overall shape of the data is consistent with an expected shape, however, it is generally unclear how to quantitatively make this determination. In practice, many analysis methods employ subjective, heuristic approaches, which complicates the validation of any ensuing results - especially as the amount and dimensionality of the data increase. Here, we present a probabilistic solution to this problem by using Bayes' rule to calculate the probability that the data has any one of several potential shapes. This probabilistic approach may be used to objectively compare how well different theories describe a dataset, identify changes between datasets, and detect features within data using a corollary method called Bayesian Inference-based Template Search (BITS); several proof-of-principle examples are provided. Altogether, this mathematical framework serves as an automated 'engine' capable of computationally executing analysis decisions currently made by visual inspection across the sciences.
Comments: 35 pages (24 Manuscript, 11 Supporting Materials), 4 Figures
Subjects: Data Analysis, Statistics and Probability (physics.data-an); Quantitative Methods (q-bio.QM)
MSC classes: 92F05 (Primary) 62F15 (secondary)
Cite as: arXiv:2109.12462 [physics.data-an]
  (or arXiv:2109.12462v3 [physics.data-an] for this version)

Submission history

From: Korak Kumar Ray [view email]
[v1] Sun, 26 Sep 2021 00:17:40 GMT (743kb)
[v2] Wed, 29 Dec 2021 08:02:51 GMT (3121kb,D)
[v3] Wed, 24 Aug 2022 05:05:36 GMT (4213kb,D)

Link back to: arXiv, form interface, contact.