We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

q-bio.QM

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Quantitative Biology > Quantitative Methods

Title: Repetition and reproduction of preclinical medical studies: taking a leaf from the plant sciences with consideration of generalised systematic errors

Abstract: Reproduction of pre-clinical results has a high failure rate. The fundamental methodology including replication ("protocol") for hypothesis testing/validation to a state allowing inference, varies within medical and plant sciences with little justification. Here, five protocols are distinguished which deal differently with systematic/random errors and vary considerably in result veracity. Aim: to compare prevalence of protocols (defined in text). Medical/plant science articles from 2017/2019 were surveyed: 713 random articles assessed for eligibility for counts: first (with p-values): 1) non-replicated; 2) global; 3) triple-result protocols; second: 4) replication-error protocol; 5) meta-analyses. Inclusion criteria: human/plant/fungal studies with categorical groups. Exclusion criteria: phased clinical trials, pilot studies, cases, reviews, technology, rare subjects, -omic studies. Abbreviated PICOS question: which protocol was evident for a main result with categorically distinct group difference(s) ? Electronic sources: Journal Citation Reports 2017/2019, Google. Triplication prevalence differed dramatically between sciences (both years p<10-16; cluster-adjusted chi-squared tests): From 320 studies (80/science/year): in 2017, 53 (66%, 95% confidence interval (C.I.) 56%:77%) and in 2019, 48 (60%, C.I. 49%:71%) plant studies had triple-result or triplicated global protocols, compared with, in both years, 4 (5%, C.I. 0.19%:9.8%) medical studies. Plant sciences had a higher prevalence of protocols more likely to counter generalised systematic errors (the most likely cause of false positives) and random error than non-replicated protocols, without suffering from serious flaws found with random-Institutes protocols. It is suggested that a triple-result (organised-reproduction) protocol, with Institute consortia, is likely to solve most problems connected with the replicability crisis.
Comments: 40 pages, 1 figure, 3 tables. Supplemental files at this https URL
Subjects: Quantitative Methods (q-bio.QM)
Cite as: arXiv:2201.10960 [q-bio.QM]
  (or arXiv:2201.10960v1 [q-bio.QM] for this version)

Submission history

From: Jeremy Clark [view email]
[v1] Wed, 26 Jan 2022 14:24:34 GMT (477kb)

Link back to: arXiv, form interface, contact.