AutoDEUQ: Automated Deep Ensemble with Uncertainty Quantification
  Deep neural networks are powerful predictors for a variety of tasks. However,
they do not capture uncertainty directly. Using neural network ensembles to
quantify uncertainty is competitive with approaches based on Bayesian neural
networks while benefiting from better computational scalability. However,
building ensembles of neural networks is a challenging task because, in
addition to choosing the right neural architecture or hyperparameters for each
member of the ensemble, there is an added cost of training each model. We
propose AutoDEUQ, an automated approach for generating an ensemble of deep
neural networks. Our approach leverages joint neural architecture and
hyperparameter search to generate ensembles. We use the law of total variance
to decompose the predictive variance of deep ensembles into aleatoric (data)
and epistemic (model) uncertainties. We show that AutoDEUQ outperforms
probabilistic backpropagation, Monte Carlo dropout, deep ensemble,
distribution-free ensembles, and hyper ensemble methods on a number of
regression benchmarks.
