DOI: Divergence-based Out-of-Distribution Indicators via Deep Generative
  Models
  To ensure robust and reliable classification results, OoD
(out-of-distribution) indicators based on deep generative models are proposed
recently and are shown to work well on small datasets. In this paper, we
conduct the first large collection of benchmarks (containing 92 dataset pairs,
which is 1 order of magnitude larger than previous ones) for existing OoD
indicators and observe that none perform well. We thus advocate that a large
collection of benchmarks is mandatory for evaluating OoD indicators. We propose
a novel theoretical framework, DOI, for divergence-based Out-of-Distribution
indicators (instead of traditional likelihood-based) in deep generative models.
Following this framework, we further propose a simple and effective OoD
detection algorithm: Single-shot Fine-tune. It significantly outperforms past
works by 5~8 in AUROC, and its performance is close to optimal. In recent, the
likelihood criterion is shown to be ineffective in detecting OoD. Single-shot
Fine-tune proposes a novel fine-tune criterion to detect OoD, by whether the
likelihood of the testing sample is improved after fine-tuning a well-trained
model on it. Fine-tune criterion is a clear and easy-following criterion, which
will lead the OoD domain into a new stage.
