Failure Identification from Unstable Log Data using Deep Learning
  The reliability of cloud platforms is of significant relevance because
society increasingly relies on complex software systems running on the cloud.
To improve it, cloud providers are automating various maintenance tasks, with
failure identification frequently being considered. The precondition for
automation is the availability of observability tools, with system logs
commonly being used. The focus of this paper is log-based failure
identification. This problem is challenging because of the instability of the
log data and the incompleteness of the explicit logging failure coverage within
the code. To address the two challenges, we present CLog as a method for
failure identification. The key idea presented herein based is on our
observation that by representing the log data as sequences of subprocesses
instead of sequences of log events, the effect of the unstable log data is
reduced. CLog introduces a novel subprocess extraction method that uses
context-aware neural network and clustering methods to extract meaningful
subprocesses. The direct modeling of log event contexts allows the
identification of failures with respect to the abrupt context changes,
addressing the challenge of insufficient logging failure coverage. Our
experimental results demonstrate that the learned subprocesses representations
reduce the instability in the input, allowing CLog to outperform the baselines
on the failure identification subproblems - 1) failure detection by 9-24% on F1
score and 2) failure type identification by 7% on the macro averaged F1 score.
Further analysis shows the existent negative correlation between the
instability in the input event sequences and the detection performance in a
model-agnostic manner.
