Anomaly Sequences Detection from Logs Based on Compression
  Mining information from logs is an old and still active research topic. In
recent years, with the rapid emerging of cloud computing, log mining becomes
increasingly important to industry. This paper focus on one major mission of
log mining: anomaly detection, and proposes a novel method for mining abnormal
sequences from large logs. Different from previous anomaly detection systems
which based on statistics, probabilities and Markov assumption, our approach
measures the strangeness of a sequence using compression. It first trains a
grammar about normal behaviors using grammar-based compression, then measures
the information quantities and densities of questionable sequences according to
incrementation of grammar length. We have applied our approach on mining some
real bugs from fine grained execution logs. We have also tested its ability on
intrusion detection using some publicity available system call traces. The
experiments show that our method successfully selects the strange sequences
which related to bugs or attacking.
