Valid prediction intervals for regression problems
  Over the last few decades, various methods have been proposed for estimating
prediction intervals in regression settings, including Bayesian methods,
ensemble methods, direct interval estimation methods and conformal prediction
methods. An important issue is the calibration of these methods: the generated
prediction intervals should have a predefined coverage level, without being
overly conservative. In this work, we review the above four classes of methods
from a conceptual and experimental point of view. Results on benchmark data
sets from various domains highlight large fluctuations in performance from one
data set to another. These observations can be attributed to the violation of
certain assumptions that are inherent to some classes of methods. We illustrate
how conformal prediction can be used as a general calibration procedure for
methods that deliver poor results without a calibration step.
