Understanding Anomaly Detection with Deep Invertible Networks through
  Hierarchies of Distributions and Features
  Deep generative networks trained via maximum likelihood on a natural image
dataset like CIFAR10 often assign high likelihoods to images from datasets with
different objects (e.g., SVHN). We refine previous investigations of this
failure at anomaly detection for invertible generative networks and provide a
clear explanation of it as a combination of model bias and domain prior:
Convolutional networks learn similar low-level feature distributions when
trained on any natural image dataset and these low-level features dominate the
likelihood. Hence, when the discriminative features between inliers and
outliers are on a high-level, e.g., object shapes, anomaly detection becomes
particularly challenging. To remove the negative impact of model bias and
domain prior on detecting high-level differences, we propose two methods,
first, using the log likelihood ratios of two identical models, one trained on
the in-distribution data (e.g., CIFAR10) and the other one on a more general
distribution of images (e.g., 80 Million Tiny Images). We also derive a novel
outlier loss for the in-distribution network on samples from the more general
distribution to further improve the performance. Secondly, using a multi-scale
model like Glow, we show that low-level features are mainly captured at early
scales. Therefore, using only the likelihood contribution of the final scale
performs remarkably well for detecting high-level feature differences of the
out-of-distribution and the in-distribution. This method is especially useful
if one does not have access to a suitable general distribution. Overall, our
methods achieve strong anomaly detection performance in the unsupervised
setting, and only slightly underperform state-of-the-art classifier-based
methods in the supervised setting. Code can be found at
https://github.com/boschresearch/hierarchical_anomaly_detection.
