We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Applications

Title: Quantifying the Reproducibility of Cell-Perturbation Experiments

Abstract: Experiments adhering to the same protocol can nonetheless lead to different conclusions, for instance, due to batch effects or lab effects. A statistical test applied to measurements from one experiment may yield a vanishingly small $p$-value, yet applying the same test to measurements from a replicate experiment may yield a large $p$-value. Recent work has highlighted this lack of reproducibility in cell-perturbation experiments. We introduce the Reproducible Sign Rate (RSR), a new reproducibility metric for settings in which each hypothesis test has two alternatives (e.g., upregulation and downregulation of gene expression). The RSR identifies the proportion of discoveries that are expected to reproduce in a future replicate. We provide conditions under which the RSR can be estimated accurately -- even when as few as two experimental replicates are available. We also provide conditions under which high RSR implies a low Type S error rate. We demonstrate the uses of RSR with experiments based on several high-throughput technologies, including L1000, Sci-Plex, and CRISPR.
Comments: Submitted to AoAS
Subjects: Applications (stat.AP)
MSC classes: 62P10 (Primary) 92C37 (Secondary)
Cite as: arXiv:2208.01745 [stat.AP]
  (or arXiv:2208.01745v1 [stat.AP] for this version)

Submission history

From: Jackson Loper [view email]
[v1] Tue, 2 Aug 2022 20:51:48 GMT (1969kb,D)

Link back to: arXiv, form interface, contact.