We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ME

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Statistics > Methodology

Title: The Most Difference in Means: A Statistic for the Strength of Null and Near-Zero Results

Abstract: Statistical insignificance does not suggest the absence of effect, yet scientists must often use null results as evidence of negligible (near-zero) effect size to falsify scientific hypotheses. Doing so must assess a result's null strength, defined as the evidence for a negligible effect size. Such an assessment would differentiate strong null results that suggest a negligible effect size from weak null results that suggest a broad range of potential effect sizes. We propose the most difference in means ($\delta_M$) as a two-sample statistic that can both quantify null strength and perform a hypothesis test for negligible effect size. To facilitate consensus when interpreting results, our statistic allows scientists to conclude that a result has negligible effect size using different thresholds with no recalculation required. To assist with selecting a threshold, $\delta_M$ can also compare null strength between related results. Both $\delta_M$ and the relative form of $\delta_M$ outperform other candidate statistics in comparing null strength. We compile broadly related results and use the relative $\delta_M$ to compare null strength across different treatments, measurement methods, and experiment models. Reporting the relative $\delta_M$ may provide a technical solution to the file drawer problem by encouraging the publication of null and near-zero results.
Subjects: Methodology (stat.ME); Quantitative Methods (q-bio.QM)
Cite as: arXiv:2201.01239 [stat.ME]
  (or arXiv:2201.01239v4 [stat.ME] for this version)

Submission history

From: Bruce Corliss [view email]
[v1] Tue, 4 Jan 2022 16:46:44 GMT (6425kb)
[v2] Thu, 6 Jan 2022 04:54:42 GMT (6425kb)
[v3] Fri, 1 Apr 2022 17:41:04 GMT (6464kb)
[v4] Tue, 24 May 2022 23:39:03 GMT (6185kb)

Link back to: arXiv, form interface, contact.