Autoencoding Under Normalization Constraints
  Likelihood is a standard estimate for outlier detection. The specific role of
the normalization constraint is to ensure that the out-of-distribution (OOD)
regime has a small likelihood when samples are learned using maximum
likelihood. Because autoencoders do not possess such a process of
normalization, they often fail to recognize outliers even when they are
obviously OOD. We propose the Normalized Autoencoder (NAE), a normalized
probabilistic model constructed from an autoencoder. The probability density of
NAE is defined using the reconstruction error of an autoencoder, which is
differently defined in the conventional energy-based model. In our model,
normalization is enforced by suppressing the reconstruction of negative
samples, significantly improving the outlier detection performance. Our
experimental results confirm the efficacy of NAE, both in detecting outliers
and in generating in-distribution samples.
