Norm-Scaling for Out-of-Distribution Detection
  Out-of-Distribution (OoD) inputs are examples that do not belong to the true
underlying distribution of the dataset. Research has shown that deep neural
nets make confident mispredictions on OoD inputs. Therefore, it is critical to
identify OoD inputs for safe and reliable deployment of deep neural nets. Often
a threshold is applied on a similarity score to detect OoD inputs. One such
similarity is angular similarity which is the dot product of latent
representation with the mean class representation. Angular similarity encodes
uncertainty, for example, if the angular similarity is less, it is less certain
that the input belongs to that class. However, we observe that, different
classes have different distributions of angular similarity. Therefore, applying
a single threshold for all classes is not ideal since the same similarity score
represents different uncertainties for different classes. In this paper, we
propose norm-scaling which normalizes the logits separately for each class.
This ensures that a single value consistently represents similar uncertainty
for various classes. We show that norm-scaling, when used with maximum softmax
probability detector, achieves 9.78% improvement in AUROC, 5.99% improvement in
AUPR and 33.19% reduction in FPR95 metrics over previous state-of-the-art
methods.
