We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.ST

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Statistics Theory

Title: Statistical inference optimized with respect to the observed sample for single or multiple comparisons

Abstract: The normalized maximum likelihood (NML) is a recent penalized likelihood that has properties that justify defining the amount of discrimination information (DI) in the data supporting an alternative hypothesis over a null hypothesis as the logarithm of an NML ratio, namely, the alternative hypothesis NML divided by the null hypothesis NML. The resulting DI, like the Bayes factor but unlike the p-value, measures the strength of evidence for an alternative hypothesis over a null hypothesis such that the probability of misleading evidence vanishes asymptotically under weak regularity conditions and such that evidence can support a simple null hypothesis. Unlike the Bayes factor, the DI does not require a prior distribution and is minimax optimal in a sense that does not involve averaging over outcomes that did not occur. Replacing a (possibly pseudo-) likelihood function with its weighted counterpart extends the scope of the DI to models for which the unweighted NML is undefined. The likelihood weights leverage side information, either in data associated with comparisons other than the comparison at hand or in the parameter value of a simple null hypothesis. Two case studies, one involving multiple populations and the other involving multiple biological features, indicate that the DI is robust to the type of side information used when that information is assigned the weight of a single observation. Such robustness suggests that very little adjustment for multiple comparisons is warranted if the sample size is at least moderate.
Comments: Typo in equation (7) of v2 corrected in equation (6) of v3; clarity improved
Subjects: Statistics Theory (math.ST); Information Theory (cs.IT); Biomolecules (q-bio.BM); Methodology (stat.ME)
MSC classes: 62Fxx
Journal reference: Bickel, D. R. (2011). A predictive approach to measuring the strength of statistical evidence for single and multiple comparisons. Canadian Journal of Statistics, 39, 610-631
DOI: 10.1002/cjs.10109
Cite as: arXiv:1010.0694 [math.ST]
  (or arXiv:1010.0694v3 [math.ST] for this version)

Submission history

From: David R. Bickel [view email]
[v1] Mon, 4 Oct 2010 20:21:49 GMT (50kb,D)
[v2] Wed, 6 Oct 2010 21:46:59 GMT (50kb,D)
[v3] Tue, 2 Nov 2010 10:33:12 GMT (51kb,D)

Link back to: arXiv, form interface, contact.