ProcessTransformer: Predictive Business Process Monitoring with
  Transformer Network
  Predictive business process monitoring focuses on predicting future
characteristics of a running process using event logs. The foresight into
process execution promises great potentials for efficient operations, better
resource management, and effective customer services. Deep learning-based
approaches have been widely adopted in process mining to address the
limitations of classical algorithms for solving multiple problems, especially
the next event and remaining-time prediction tasks. Nevertheless, designing a
deep neural architecture that performs competitively across various tasks is
challenging as existing methods fail to capture long-range dependencies in the
input sequences and perform poorly for lengthy process traces. In this paper,
we propose ProcessTransformer, an approach for learning high-level
representations from event logs with an attention-based network. Our model
incorporates long-range memory and relies on a self-attention mechanism to
establish dependencies between a multitude of event sequences and corresponding
outputs. We evaluate the applicability of our technique on nine real event
logs. We demonstrate that the transformer-based model outperforms several
baselines of prior techniques by obtaining on average above 80% accuracy for
the task of predicting the next activity. Our method also perform
competitively, compared to baselines, for the tasks of predicting event time
and remaining time of a running case
