FlexParser -- the adaptive log file parser for continuous results in a
  changing world
  Any modern system writes events into files, called log files. Those contain
crucial information which are subject to various analyses. Examples range from
cybersecurity, intrusion detection over usage analyses to trouble shooting.
Before data analysis is possible, desired information needs to be extracted
first out of the semi-structured log messages. State-of-the-art event parsing
often assumes static log events. However, any modern system is updated
consistently and with updates also log file structures can change. We call
those changes "mutation" and study parsing performance for different mutation
cases. Latest research discovers mutations using anomaly detection post mortem,
however, does not cover actual continuous parsing. Thus, we propose a novel and
flexible parser, called FlexParser, which can extract desired values despite
gradual changes in the log messages. It implies basic text preprocessing
followed by a supervised Deep Learning method. We train a stateful LSTM on
parsing one event per data set. Statefulness enforces the model to learn log
message structures across several examples. Our model was tested on seven
different, publicly available log file data sets and various kinds of
mutations. Exhibiting an average F1-Score of 0.98, it outperforms other Deep
Learning methods as well as state-of-the-art unsupervised parsers.
