A statistical framework for efficient out of distribution detection in
  deep neural networks
  Background. Commonly, Deep Neural Networks (DNNs) generalize well on samples
drawn from a distribution similar to that of the training set. However, DNNs'
predictions are brittle and unreliable when the test samples are drawn from a
dissimilar distribution. This is a major concern for deployment in real-world
applications, where such behavior may come at a considerable cost, such as
industrial production lines, autonomous vehicles, or healthcare applications.
Contributions. We frame Out Of Distribution (OOD) detection in DNNs as a
statistical hypothesis testing problem. Tests generated within our proposed
framework combine evidence from the entire network. Unlike previous OOD
detection heuristics, this framework returns a $p$-value for each test sample.
It is guaranteed to maintain the Type I Error (T1E - incorrectly predicting OOD
for an actual in-distribution sample) for test data. Moreover, this allows to
combine several detectors while maintaining the T1E. Building on this
framework, we suggest a novel OOD procedure based on low-order statistics. Our
method achieves comparable or better results than state-of-the-art methods on
well-accepted OOD benchmarks, without retraining the network parameters or
assuming prior knowledge on the test distribution -- and at a fraction of the
computational cost.
