When does Active Learning Work?
  Active Learning (AL) methods seek to improve classifier performance when
labels are expensive or scarce. We consider two central questions: Where does
AL work? How much does it help? To address these questions, a comprehensive
experimental simulation study of Active Learning is presented. We consider a
variety of tasks, classifiers and other AL factors, to present a broad
exploration of AL performance in various settings. A precise way to quantify
performance is needed in order to know when AL works. Thus we also present a
detailed methodology for tackling the complexities of assessing AL performance
in the context of this experimental study.
