We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

econ.EM

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Economics > Econometrics

Title: Incentive-Compatible Critical Values

Abstract: Statistically significant results are more rewarded than insignificant ones, so researchers have the incentive to pursue statistical significance. Such p-hacking reduces the informativeness of hypothesis tests by making significant results much more common than they are supposed to be in the absence of true significance. To address this problem, we construct critical values of test statistics such that, if these values are used to determine significance, and if researchers optimally respond to these new significance standards, then significant results occur with the desired frequency. Such incentive-compatible critical values allow for p-hacking so they are larger than classical critical values. Using evidence from the social and medical sciences, we find that the incentive-compatible critical value for any test and any significance level is the classical critical value for the same test with approximately one fifth of the significance level -- a form of Bonferroni correction. For instance, for a z-test with a significance level of 5%, the incentive-compatible critical value is 2.31 instead of 1.65 if the test is one-sided and 2.57 instead of 1.96 if the test is two-sided.
Subjects: Econometrics (econ.EM); Methodology (stat.ME)
Cite as: arXiv:2005.04141 [econ.EM]
  (or arXiv:2005.04141v3 [econ.EM] for this version)

Submission history

From: Pascal Michaillat [view email]
[v1] Fri, 8 May 2020 16:37:11 GMT (1634kb,D)
[v2] Thu, 23 Dec 2021 18:26:56 GMT (142kb,D)
[v3] Tue, 11 Jan 2022 03:15:17 GMT (144kb,D)
[v4] Thu, 16 Jun 2022 02:58:47 GMT (144kb,D)

Link back to: arXiv, form interface, contact.