A Semi-Supervised Approach for Abnormal Event Prediction on Large
  Operational Network Time-Series Data
  Large network logs, recording multivariate time series generated from
heterogeneous devices and sensors in a network, can often reveal important
information about abnormal activities, such as network intrusions and device
malfunctions. Existing machine learning methods for anomaly detection on
multivariate time series typically assume that 1) normal sequences would have
consistent behavior for training unsupervised models, or 2) require a large set
of labeled normal and abnormal sequences for supervised models. However, in
practice, normal network activities can demonstrate significantly varying
sequence patterns (e.g., before and after rerouting partial network traffic).
Also, the recorded abnormal events can be sparse. This paper presents a novel
semi-supervised method that efficiently captures dependencies between network
time series and across time points to generate meaningful representations of
network activities for predicting abnormal events. The method can use the
limited labeled data to explicitly learn separable embedding space for normal
and abnormal samples and effectively leverage unlabeled data to handle training
data scarcity. The experiments demonstrate that our approach significantly
outperformed state-of-the-art approaches for event detection on a large
real-world network log.
