Model2Detector: Widening the Information Bottleneck for
  Out-of-Distribution Detection using a Handful of Gradient Steps
  Out-of-distribution detection is an important capability that has long eluded
vanilla neural networks. Deep Neural networks (DNNs) tend to generate
over-confident predictions when presented with inputs that are significantly
out-of-distribution (OOD). This can be dangerous when employing machine
learning systems in the wild as detecting attacks can thus be difficult. Recent
advances inference-time out-of-distribution detection help mitigate some of
these problems. However, existing methods can be restrictive as they are often
computationally expensive. Additionally, these methods require training of a
downstream detector model which learns to detect OOD inputs from
in-distribution ones. This, therefore, adds latency during inference. Here, we
offer an information theoretic perspective on why neural networks are
inherently incapable of OOD detection. We attempt to mitigate these flaws by
converting a trained model into a an OOD detector using a handful of steps of
gradient descent. Our work can be employed as a post-processing method whereby
an inference-time ML system can convert a trained model into an OOD detector.
Experimentally, we show how our method consistently outperforms the
state-of-the-art in detection accuracy on popular image datasets while also
reducing computational complexity.
