Enhanced Isotropy Maximization Loss: Seamless and High-Performance
  Out-of-Distribution Detection Simply Replacing the SoftMax Loss
  Current out-of-distribution detection approaches usually present special
requirements (e.g., collecting outlier data and hyperparameter validation) and
produce side effects (e.g., classification accuracy drop and slow/inefficient
inferences). Recently, entropic out-of-distribution detection has been proposed
as a seamless approach (i.e., a solution that avoids all previously mentioned
drawbacks). The entropic out-of-distribution detection solution uses the IsoMax
loss for training and the entropic score for out-of-distribution detection. The
IsoMax loss works as a drop-in replacement of the SoftMax loss (i.e., the
combination of the output linear layer, the SoftMax activation, and the
cross-entropy loss) because swapping the SoftMax loss with the IsoMax loss
requires no changes in the model's architecture or training
procedures/hyperparameters. In this paper, we perform what we call an
isometrization of the distances used in the IsoMax loss. Additionally, we
propose replacing the entropic score with the minimum distance score.
Experiments showed that these modifications significantly increase
out-of-distribution detection performance while keeping the solution seamless.
Besides being competitive with or outperforming all major current approaches,
the proposed solution avoids all their current limitations, in addition to
being much easier to use because only a simple loss replacement for training
the neural network is required. The code to replace the SoftMax loss with the
IsoMax+ loss and reproduce the results is available at
https://github.com/dlmacedo/entropic-out-of-distribution-detection.
