Clustered Hierarchical Anomaly and Outlier Detection Algorithms
  Anomaly and outlier detection is a long-standing problem in machine learning.
In some cases, anomaly detection is easy, such as when data are drawn from
well-characterized distributions such as the Gaussian. However, when data
occupy high-dimensional spaces, anomaly detection becomes more difficult. We
present CLAM (Clustered Learning of Approximate Manifolds), a manifold mapping
technique in any metric space. CLAM begins with a fast hierarchical clustering
technique and then induces a graph from the cluster tree, based on overlapping
clusters as selected using several geometric and topological features. Using
these graphs, we implement CHAODA (Clustered Hierarchical Anomaly and Outlier
Detection Algorithms), exploring various properties of the graphs and their
constituent clusters to find outliers. CHAODA employs a form of transfer
learning based on a training set of datasets, and applies this knowledge to a
separate test set of datasets of different cardinalities, dimensionalities, and
domains. On 24 publicly available datasets, we compare CHAODA (by measure of
ROC AUC) to a variety of state-of-the-art unsupervised anomaly-detection
algorithms. Six of the datasets are used for training. CHAODA outperforms other
approaches on 16 of the remaining 18 datasets. CLAM and CHAODA scale to large,
high-dimensional "big data" anomaly-detection problems, and generalize across
datasets and distance functions. Source code to CLAM and CHAODA are freely
available on GitHub at https://github.com/URI-ABD/clam.
