DeCorus: Hierarchical Multivariate Anomaly Detection at Cloud-Scale
  Multivariate anomaly detection can be used to identify outages within large
volumes of telemetry data for computing systems. However, developing an
efficient anomaly detector that can provide users with relevant information is
a challenging problem. We introduce our approach to hierarchical multivariate
anomaly detection called DeCorus, a statistical multivariate anomaly detector
which achieves linear complexity. It extends standard statistical techniques to
improve their ability to find relevant anomalies within noisy signals and makes
use of types of domain knowledge that system operators commonly possess to
compute system-level anomaly scores. We describe the implementation of DeCorus
an online log anomaly detection tool for network device syslog messages
deployed at a cloud service provider. We use real-world data sets that consist
of $1.5$ billion network device syslog messages and hundreds of incident
tickets to characterize the performance of DeCorus and compare its ability to
detect incidents with five alternative anomaly detectors. While DeCorus
outperforms the other anomaly detectors, all of them are challenged by our data
set. We share how DeCorus provides value in the field and how we plan to
improve its incident detection accuracy.
