Joint Distribution across Representation Space for Out-of-Distribution
  Detection
  Deep neural networks (DNNs) have become a key part of many modern software
applications. After training and validating, the DNN is deployed as an
irrevocable component and applied in real-world scenarios. Although most DNNs
are built meticulously with huge volumes of training data, data in the real
world still remain unknown to the DNN model, which leads to the crucial
requirement of runtime out-of-distribution (OOD) detection. However, many
existing approaches 1) need OOD data for classifier training or parameter
tuning, or 2) simply combine the scores of each hidden layer as an ensemble of
features for OOD detection. In this paper, we present a novel outlook on
in-distribution data in a generative manner, which takes their latent features
generated from each hidden layer as a joint distribution across representation
spaces. Since only the in-distribution latent features are comprehensively
understood in representation space, the internal difference between
in-distribution and OOD data can be naturally revealed without the intervention
of any OOD data. Specifically, We construct a generative model, called Latent
Sequential Gaussian Mixture (LSGM), to depict how the in-distribution latent
features are generated in terms of the trace of DNN inference across
representation spaces. We first construct the Gaussian Mixture Model (GMM)
based on in-distribution latent features for each hidden layer, and then
connect GMMs via the transition probabilities of the inference traces.
Experimental evaluations on popular benchmark OOD datasets and models validate
the superiority of the proposed method over the state-of-the-art methods in OOD
detection.
