Accurate Uncertainty Estimation and Decomposition in Ensemble Learning
  Ensemble learning is a standard approach to building machine learning systems
that capture complex phenomena in real-world data. An important aspect of these
systems is the complete and valid quantification of model uncertainty. We
introduce a Bayesian nonparametric ensemble (BNE) approach that augments an
existing ensemble model to account for different sources of model uncertainty.
BNE augments a model's prediction and distribution functions using Bayesian
nonparametric machinery. It has a theoretical guarantee in that it robustly
estimates the uncertainty patterns in the data distribution, and can decompose
its overall predictive uncertainty into distinct components that are due to
different sources of noise and error. We show that our method achieves accurate
uncertainty estimates under complex observational noise, and illustrate its
real-world utility in terms of uncertainty decomposition and model bias
detection for an ensemble in predict air pollution exposures in Eastern
Massachusetts, USA.
