We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.GT

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computer Science and Game Theory

Title: Are You Smarter Than a Random Expert? The Robust Aggregation of Substitutable Signals

Abstract: The problem of aggregating expert forecasts is ubiquitous in fields as wide-ranging as machine learning, economics, climate science, and national security. Despite this, our theoretical understanding of this question is fairly shallow. This paper initiates the study of forecast aggregation in a context where experts' knowledge is chosen adversarially from a broad class of information structures. While in full generality it is impossible to achieve a nontrivial performance guarantee, we show that doing so is possible under a condition on the experts' information structure that we call \emph{projective substitutes}. The projective substitutes condition is a notion of informational substitutes: that there are diminishing marginal returns to learning the experts' signals. We show that under the projective substitutes condition, taking the average of the experts' forecasts improves substantially upon the strategy of trusting a random expert. We then consider a more permissive setting, in which the aggregator has access to the prior. We show that by averaging the experts' forecasts and then \emph{extremizing} the average by moving it away from the prior by a constant factor, the aggregator's performance guarantee is substantially better than is possible without knowledge of the prior. Our results give a theoretical grounding to past empirical research on extremization and help give guidance on the appropriate amount to extremize.
Comments: 23 pages, 2 figures
Subjects: Computer Science and Game Theory (cs.GT); Machine Learning (cs.LG)
Cite as: arXiv:2111.03153 [cs.GT]
  (or arXiv:2111.03153v2 [cs.GT] for this version)

Submission history

From: Eric Neyman [view email]
[v1] Thu, 4 Nov 2021 20:50:30 GMT (46kb,D)
[v2] Wed, 23 Feb 2022 00:09:40 GMT (44kb,D)

Link back to: arXiv, form interface, contact.