On the Performance Analysis of the Adversarial System Variant
  Approximation Method to Quantify Process Model Generalization
  Process mining algorithms discover a process model from an event log. The
resulting process model is supposed to describe all possible event sequences of
the underlying system. Generalization is a process model quality dimension of
interest. A generalization metric should quantify the extent to which a process
model represents the observed event sequences contained in the event log and
the unobserved event sequences of the system. Most of the available metrics in
the literature cannot properly quantify the generalization of a process model.
A recently published method [1] called Adversarial System Variant Approximation
leverages Generative Adversarial Networks to approximate the underlying event
sequence distribution of a system from an event log. While this method
demonstrated performance gains over existing methods in measuring the
generalization of process models, its experimental evaluations have been
performed under ideal conditions. This paper experimentally investigates the
performance of Adversarial System Variant Approximation under non-ideal
conditions such as biased and limited event logs. Moreover, experiments are
performed to investigate the originally proposed sampling hyperparameter value
of the method on its performance to measure the generalization. The results
confirm the need to raise awareness about the working conditions of the
Adversarial System Variant Approximation method. The outcomes of this paper
also serve to initiate future research directions.
  [1] Theis, Julian, and Houshang Darabi. "Adversarial System Variant
Approximation to Quantify Process Model Generalization." IEEE Access 8 (2020):
194410-194427.
