A Simulation Study of Bandit Algorithms to Address External Validity of
  Software Fault Prediction
  Various software fault prediction models and techniques for building
algorithms have been proposed. Many studies have compared and evaluated them to
identify the most effective ones. However, in most cases, such models and
techniques do not have the best performance on every dataset. This is because
there is diversity of software development datasets, and therefore, there is a
risk that the selected model or technique shows bad performance on a certain
dataset. To avoid selecting a low accuracy model, we apply bandit algorithms to
predict faults. Consider a case where player has 100 coins to bet on several
slot machines. Ordinary usage of software fault prediction is analogous to the
player betting all 100 coins in one slot machine. In contrast, bandit
algorithms bet one coin on each machine (i.e., use prediction models)
step-by-step to seek the best machine. In the experiment, we developed an
artificial dataset that includes 100 modules, 15 of which include faults. Then,
we developed various artificial fault prediction models and selected them
dynamically using bandit algorithms. The Thomson sampling algorithm showed the
best or second-best prediction performance compared with using only one
prediction model.
