We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.AP

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Statistics > Machine Learning

Title: Machine-Learning Tests for Effects on Multiple Outcomes

Abstract: In this paper we present tools for applied researchers that re-purpose off-the-shelf methods from the computer-science field of machine learning to create a "discovery engine" for data from randomized controlled trials (RCTs). The applied problem we seek to solve is that economists invest vast resources into carrying out RCTs, including the collection of a rich set of candidate outcome measures. But given concerns about inference in the presence of multiple testing, economists usually wind up exploring just a small subset of the hypotheses that the available data could be used to test. This prevents us from extracting as much information as possible from each RCT, which in turn impairs our ability to develop new theories or strengthen the design of policy interventions. Our proposed solution combines the basic intuition of reverse regression, where the dependent variable of interest now becomes treatment assignment itself, with methods from machine learning that use the data themselves to flexibly identify whether there is any function of the outcomes that predicts (or has signal about) treatment group status. This leads to correctly-sized tests with appropriate $p$-values, which also have the important virtue of being easy to implement in practice. One open challenge that remains with our work is how to meaningfully interpret the signal that these methods find.
Comments: Minor update: new abstract, introduction, and section on representing distribution differences
Subjects: Machine Learning (stat.ML); Econometrics (econ.EM); Applications (stat.AP); Methodology (stat.ME)
Cite as: arXiv:1707.01473 [stat.ML]
  (or arXiv:1707.01473v2 [stat.ML] for this version)

Submission history

From: Jann Spiess [view email]
[v1] Wed, 5 Jul 2017 17:13:08 GMT (840kb,D)
[v2] Thu, 9 May 2019 18:10:56 GMT (866kb,D)

Link back to: arXiv, form interface, contact.