A Multi-Objective Anytime Rule Mining System to Ease Iterative Feedback
  from Domain Experts
  Data extracted from software repositories is used intensively in Software
Engineering research, for example, to predict defects in source code. In our
research in this area, with data from open source projects as well as an
industrial partner, we noticed several shortcomings of conventional data mining
approaches for classification problems: (1) Domain experts' acceptance is of
critical importance, and domain experts can provide valuable input, but it is
hard to use this feedback. (2) The evaluation of the model is not a simple
matter of calculating AUC or accuracy. Instead, there are multiple objectives
of varying importance, but their importance cannot be easily quantified.
Furthermore, the performance of the model cannot be evaluated on a per-instance
level in our case, because it shares aspects with the set cover problem. To
overcome these problems, we take a holistic approach and develop a rule mining
system that simplifies iterative feedback from domain experts and can easily
incorporate the domain-specific evaluation needs. A central part of the system
is a novel multi-objective anytime rule mining algorithm. The algorithm is
based on the GRASP-PR meta-heuristic but extends it with ideas from several
other approaches. We successfully applied the system in the industrial context.
In the current article, we focus on the description of the algorithm and the
concepts of the system. We provide an implementation of the system for reuse.
