PI3NN: Out-of-distribution-aware prediction intervals from three neural
  networks
  We propose a novel prediction interval (PI) method for uncertainty
quantification, which addresses three major issues with the state-of-the-art PI
methods. First, existing PI methods require retraining of neural networks (NNs)
for every given confidence level and suffer from the crossing issue in
calculating multiple PIs. Second, they usually rely on customized loss
functions with extra sensitive hyperparameters for which fine tuning is
required to achieve a well-calibrated PI. Third, they usually underestimate
uncertainties of out-of-distribution (OOD) samples leading to over-confident
PIs. Our PI3NN method calculates PIs from linear combinations of three NNs,
each of which is independently trained using the standard mean squared error
loss. The coefficients of the linear combinations are computed using
root-finding algorithms to ensure tight PIs for a given confidence level. We
theoretically prove that PI3NN can calculate PIs for a series of confidence
levels without retraining NNs and it completely avoids the crossing issue.
Additionally, PI3NN does not introduce any unusual hyperparameters resulting in
a stable performance. Furthermore, we address OOD identification challenge by
introducing an initialization scheme which provides reasonably larger PIs of
the OOD samples than those of the in-distribution samples. Benchmark and
real-world experiments show that our method outperforms several
state-of-the-art approaches with respect to predictive uncertainty quality,
robustness, and OOD samples identification.
