ReAct: Out-of-distribution Detection With Rectified Activations
  Out-of-distribution (OOD) detection has received much attention lately due to
its practical importance in enhancing the safe deployment of neural networks.
One of the primary challenges is that models often produce highly confident
predictions on OOD data, which undermines the driving principle in OOD
detection that the model should only be confident about in-distribution
samples. In this work, we propose ReAct--a simple and effective technique for
reducing model overconfidence on OOD data. Our method is motivated by novel
analysis on internal activations of neural networks, which displays highly
distinctive signature patterns for OOD distributions. Our method can generalize
effectively to different network architectures and different OOD detection
scores. We empirically demonstrate that ReAct achieves competitive detection
performance on a comprehensive suite of benchmark datasets, and give
theoretical explication for our method's efficacy. On the ImageNet benchmark,
ReAct reduces the false positive rate (FPR95) by 25.05% compared to the
previous best method.
