3.4. Word Embeddings 3.5. Language and Process As neural networks operate on real-valued data, words or any other categori- cal input must be appropriately encoded. One option for this is to use ”one-hot” encodings, but this leads to input vectors and neural net layers whose size (the dimensionality m in Sec. 3.3) is the number of distinct words or categories. Word embeddings avoid this problem and allows arbitrary input vector and neural net- work sizes. Words are mapped into an n-dimensional ”embedding” space R v × m using an ”embedding matrix”, which is essentially a look-up matrix of dimensions v × m where v is the size of the vocabulary and m is the chosen dimensionality of the embedding space and hence the size of each LSTM hidden layer (cf. Section 3.3). The input layer in Fig. 1 is an embedding lookup function yields a numeric vector from$_{R}$ m for each word in the vocabulary, which forms the input x$_{t}$ in Eq. 1, 2 and 5 for the first LSTM layer. The embedding matrix is also a trainable parameter that is learned during training. The output layer produces a probability distribution over the words in the vocabulary. Each output ”box” in Fig. 1 represents a softmax layer with v individual cells. The word with the highest probability is selected as the predicted word and compared to the target. Referring to the XES standard (XES Working Group, 2016), we define events jointly by the name of the executing activity (e.g. ”Pay Invoice”, ”Repair Wid- get”), the lifecycle transition (e.g. ”Start”, ”Complete”, ”Schedule”) and, for some experimental settings, the organizational resource, role or group (e.g. ”Jane Doe”, ”Tester”, ”Accounting”). An example of an event is ”Repair Widget—Start—Jane Doe”. Our ”vocabulary” is the set of unique events of this form contained in a log. As natural language is constrained by grammatical and morphological rules, such as noun and verb agreement for plurals in English, process event sequences are 10