Table 2: Characteristics of datasets used in experimental evaluation. Event numbers for partial BPI2012 logs do not add up to that of corresponding complete log due to end-of-case marker events added to each trace. 5.2. Evaluation Method There are three random influences on prediction performance. First, the train- able parameters are initialized randomly and different initial values may lead to different outcomes (e.g. convergence to local optima). Second, the traces in a log are in a particular order but this is an arbitrary order. Because the LSTM cells maintain state, the order in which traces are used to train the network may have an effect on the outcome. Finally, the selection of the training dataset itself has an influence, and the results obtained with a particular sample may not generalize to others. Both BPI 2012 and 2013 datasets are sampled from running systems. To address these stochastic influences, we perform 10-fold cross-validation (Hastie et al., 2009). Comparing the prediction performance on an independent validation sample to the prediction performance on the training sample assesses the general- izability to similar datasets. Specifically, it examines whether the RNN overfits the training sample, i.e. exploits idiosyncrasies in the training sample, and what type of performance can realistically be expected on a dataset with similar characteristics. Note that Breuker et al. (2016) do not cross-validate the results of their stochastic EM-based approach; a fair comparison is therefore to our training results. We report the prediction precision, defined as the proportion of correct predic- 16 Dataset Number of unique event types (”vocabulary size”) Number of events w/ res. info BPI2013.Incidents 14 3133 65533 BPI2013.Problems 8 64 9011 BPI2012 (completion events) 24 877 164506 BPI2012 (all events) 37 1349 262200 BPI2012.W (completion events) 7 264 72413 BPI2012.W (all events) 7 736 169507 BPI2012.A 11 302 60849 BPI2012.O 8 313 31244