Instance-Based Uncertainty Estimation for Gradient-Boosted Regression
  Trees
  We propose Instance-Based Uncertainty estimation for Gradient-boosted
regression trees~(IBUG), a simple method for extending any GBRT point predictor
to produce probabilistic predictions. IBUG computes a non-parametric
distribution around a prediction using the k-nearest training instances, where
distance is measured with a tree-ensemble kernel. The runtime of IBUG depends
on the number of training examples at each leaf in the ensemble, and can be
improved by sampling trees or training instances. Empirically, we find that
IBUG achieves similar or better performance than the previous state-of-the-art
across 22 benchmark regression datasets. We also find that IBUG can achieve
improved probabilistic performance by using different base GBRT models, and can
more flexibly model the posterior distribution of a prediction than competing
methods. We also find that previous methods suffer from poor probabilistic
calibration on some datasets, which can be mitigated using a scalar factor
tuned on the validation data.
