Mean-Shifted Contrastive Loss for Anomaly Detection
  Deep anomaly detection methods learn representations that separate between
normal and anomalous samples. Very effective representations are obtained when
powerful externally trained feature extractors (e.g. ResNets pre-trained on
ImageNet) are fine-tuned on the training data which consists of normal samples
and no anomalies. However, this is a difficult task that can suffer from
catastrophic collapse, i.e. it is prone to learning trivial and non-specific
features. In this paper, we propose a new loss function which can overcome
failure modes of both center-loss and contrastive-loss methods. Furthermore, we
combine it with a confidence-invariant angular center loss, which replaces the
Euclidean distance used in previous work, that was sensitive to prediction
confidence. Our improvements yield a new anomaly detection approach, based on
$\textit{Mean-Shifted Contrastive Loss}$, which is both more accurate and less
sensitive to catastrophic collapse than previous methods. Our method achieves
state-of-the-art anomaly detection performance on multiple benchmarks including
$97.5\%$ ROC-AUC on the CIFAR-10 dataset.
