Are You Smarter Than a Random Expert? The Robust Aggregation of
  Substitutable Signals
  The problem of aggregating expert forecasts is ubiquitous in fields as
wide-ranging as machine learning, economics, climate science, and national
security. Despite this, our theoretical understanding of this question is
fairly shallow. This paper initiates the study of forecast aggregation in a
context where experts' knowledge is chosen adversarially from a broad class of
information structures. While in full generality it is impossible to achieve a
nontrivial performance guarantee, we show that doing so is possible under a
condition on the experts' information structure that we call \emph{projective
substitutes}. The projective substitutes condition is a notion of informational
substitutes: that there are diminishing marginal returns to learning the
experts' signals. We show that under the projective substitutes condition,
taking the average of the experts' forecasts improves substantially upon the
strategy of trusting a random expert. We then consider a more permissive
setting, in which the aggregator has access to the prior. We show that by
averaging the experts' forecasts and then \emph{extremizing} the average by
moving it away from the prior by a constant factor, the aggregator's
performance guarantee is substantially better than is possible without
knowledge of the prior. Our results give a theoretical grounding to past
empirical research on extremization and help give guidance on the appropriate
amount to extremize.
