Adaptive, Distribution-Free Prediction Intervals for Deep Networks
  The machine learning literature contains several constructions for prediction
intervals that are intuitively reasonable but ultimately ad-hoc in that they do
not come with provable performance guarantees. We present methods from the
statistics literature that can be used efficiently with neural networks under
minimal assumptions with guaranteed performance. We propose a neural network
that outputs three values instead of a single point estimate and optimizes a
loss function motivated by the standard quantile regression loss. We provide
two prediction interval methods with finite sample coverage guarantees solely
under the assumption that the observations are independent and identically
distributed. The first method leverages the conformal inference framework and
provides average coverage. The second method provides a new, stronger guarantee
by conditioning on the observed data. Lastly, our loss function does not
compromise the predictive accuracy of the network like other prediction
interval methods. We demonstrate the ease of use of our procedures as well as
its improvements over other methods on both simulated and real data. As most
deep networks can easily be modified by our method to output predictions with
valid prediction intervals, its use should become standard practice, much like
reporting standard errors along with mean estimates.
