Anomaly Detection in Predictive Maintenance: A New Evaluation Framework
  for Temporal Unsupervised Anomaly Detection Algorithms
  The research in anomaly detection lacks a unified definition of what
represents an anomalous instance. Discrepancies in the nature itself of an
anomaly lead to multiple paradigms of algorithms design and experimentation.
Predictive maintenance is a special case, where the anomaly represents a
failure that must be prevented. Related time-series research as outlier and
novelty detection or time-series classification does not apply to the concept
of an anomaly in this field, because they are not single points which have not
been seen previously and may not be precisely annotated. Moreover, due to the
lack of annotated anomalous data, many benchmarks are adapted from supervised
scenarios.
  To address these issues, we generalise the concept of positive and negative
instances to intervals to be able to evaluate unsupervised anomaly detection
algorithms. We also preserve the imbalance scheme for evaluation through the
proposal of the Preceding Window ROC, a generalisation for the calculation of
ROC curves for time-series scenarios. We also adapt the mechanism from a
established time-series anomaly detection benchmark to the proposed
generalisations to reward early detection. Therefore, the proposal represents a
flexible evaluation framework for the different scenarios. To show the
usefulness of this definition, we include a case study of Big Data algorithms
with a real-world time-series problem provided by the company ArcelorMittal,
and compare the proposal with an evaluation method.
