DEUP: Direct Epistemic Uncertainty Prediction
  Epistemic Uncertainty (EU) is the part of out-of-sample prediction error due
to the lack of knowledge of the learner. While existing work focuses on model
variance as a proxy of EU, we propose a principled framework for directly
estimating it by learning to predict the generalization error and subtracting
an estimate of aleatoric uncertainty, i.e., intrinsic unpredictability, using a
secondary learner. We discuss the merits of this novel interpretation of EU,
and highlight how it differs from variance-based proxies of EU and addresses
their shortcomings. Our framework, Direct Epistemic Uncertainty Prediction
(DEUP) is particularly interesting in interactive learning environments, where
the learner is allowed to acquire novel examples in each round. Through a wide
set of experiments, we illustrate how existing methods in sequential model
optimization can be improved with EU estimates from DEUP, and how DEUP can be
used to drive exploration in reinforcement learning. We also evaluate the
quality of uncertainty estimates from DEUP for probabilistic image
classification and predicting synergies of drug combinations.
