Simple and Effective Prevention of Mode Collapse in Deep One-Class
  Classification
  Anomaly detection algorithms find extensive use in various fields. This area
of research has recently made great advances thanks to deep learning. A recent
method, the deep Support Vector Data Description (deep SVDD), which is inspired
by the classic kernel-based Support Vector Data Description (SVDD), is capable
of simultaneously learning a feature representation of the data and a
data-enclosing hypersphere. The method has shown promising results in both
unsupervised and semi-supervised settings. However, deep SVDD suffers from
hypersphere collapse -- also known as mode collapse, if the architecture of the
model does not comply with certain architectural constraints, e.g. the removal
of bias terms. These constraints limit the adaptability of the model and in
some cases, may affect the model performance due to learning sub-optimal
features. In this work, we consider two regularizers to prevent hypersphere
collapse in deep SVDD. The first regularizer is based on injecting random noise
via the standard cross-entropy loss. The second regularizer penalizes the
minibatch variance when it becomes too small. Moreover, we introduce an
adaptive weighting scheme to control the amount of penalization between the
SVDD loss and the respective regularizer. Our proposed regularized variants of
deep SVDD show encouraging results and outperform a prominent state-of-the-art
method on a setup where the anomalies have no apparent geometrical structure.
