Probabilistic Robust Autoencoders for Anomaly Detection
  Anomalies (or outliers) are prevalent in real-world empirical observations
and potentially mask important underlying structures. Accurate identification
of anomalous samples is crucial for the success of downstream data analysis
tasks. To automatically identify anomalies, we propose Probabilistic Robust
AutoEncoder (PRAE). PRAE is designed to simultaneously remove outliers and
identify a low-dimensional representation for the inlier samples. We first
present the Robust AutoEncoder (RAE) intractable objective as a minimization
problem for splitting the data to inlier samples from which a low dimensional
representation is learned via an AutoEncoder (AE), and anomalous (outlier)
samples that are excluded as they do not fit the low dimensional
representation. RAE minimizes the autoencoder's reconstruction error while
incorporating as many samples as possible. This could be formulated via
regularization by subtracting from the reconstruction term an $\ell_0$ norm
counting the number of selected samples. Unfortunately, this leads to an
intractable combinatorial problem. Therefore, we propose two probabilistic
relaxations of RAE, which are differentiable and alleviate the need for a
combinatorial search. We prove that the solution to the PRAE problem is
equivalent to the solution of RAE. We use synthetic data to show that PRAE can
accurately remove outliers in a wide range of contamination frequencies.
Finally, we demonstrate that using PRAE for anomaly detection leads to
state-of-the-art results on various benchmark datasets.
