Evaluating Conformance Measures in Process Mining using Conformance
  Propositions (Extended version)
  Process mining sheds new light on the relationship between process models and
real-life processes. Process discovery can be used to learn process models from
event logs. Conformance checking is concerned with quantifying the quality of a
business process model in relation to event data that was logged during the
execution of the business process. There exist different categories of
conformance measures. Recall, also called fitness, is concerned with
quantifying how much of the behavior that was observed in the event log fits
the process model. Precision is concerned with quantifying how much behavior a
process model allows for that was never observed in the event log.
Generalization is concerned with quantifying how well a process model
generalizes to behavior that is possible in the business process but was never
observed in the event log. Many recall, precision, and generalization measures
have been developed throughout the years, but they are often defined in an
ad-hoc manner without formally defining the desired properties up front. To
address these problems, we formulate 21 conformance propositions and we use
these propositions to evaluate current and existing conformance measures. The
goal is to trigger a discussion by clearly formulating the challenges and
requirements (rather than proposing new measures). Additionally, this paper
serves as an overview of the conformance checking measures that are available
in the process mining area.
