What Averages Do Not Tell -- Predicting Real Life Processes with
  Sequential Deep Learning
  Deep Learning is proven to be an effective tool for modeling sequential data
as shown by the success in Natural Language, Computer Vision and Signal
Processing. Process Mining concerns discovering insights on business processes
from their execution data that are logged by supporting information systems.
The logged data (event log) is formed of event sequences (traces) that
correspond to executions of a process. Many Deep Learning techniques have been
successfully adapted for predictive Process Mining that aims to predict process
outcomes, remaining time, the next event, or even the suffix of running traces.
Traces in Process Mining are multimodal sequences and very differently
structured than natural language sentences or images. This may require a
different approach to processing. So far, there has been little focus on these
differences and the challenges introduced. Looking at suffix prediction as the
most challenging of these tasks, the performance of Deep Learning models was
evaluated only on average measures and for a small number of real-life event
logs. Comparing the results between papers is difficult due to different
pre-processing and evaluation strategies. Challenges that may be relevant are
the skewness of trace-length distribution and the skewness of the activity
distribution in real-life event logs. We provide an end-to-end framework which
enables to compare the performance of seven state-of-the-art sequential
architectures in common settings. Results show that sequence modeling still has
a lot of room for improvement for majority of the more complex datasets.
Further research and insights are required to get consistent performance not
just in average measures but additionally over all the prefixes.
