A Comparative Evaluation of Quantification Methods
  Quantification represents the problem of predicting class distributions in a
dataset. It also represents a growing research field in supervised machine
learning, for which a large variety of different algorithms has been proposed
in recent years. However, a comprehensive empirical comparison of
quantification methods that supports algorithm selection is not available yet.
In this work, we close this research gap by conducting a thorough empirical
performance comparison of 24 different quantification methods on 40 data sets,
considering binary as well as multiclass quantification settings. We observe
that no single algorithm generally outperforms all competitors, but identify a
group of methods including the Median Sweep and the DyS framework that performs
best in the binary setting. We also find that tuning the underlying classifiers
has in most cases only a limited impact on the quantification performance. For
the multiclass setting, we observe that a different, broad group of algorithms
yields good performance, including the Generalized Probabilistic Adjusted
Count, the readme method, the energy distance minimization method, the EM
algorithm for quantification, and Friedman's method. More generally, we find
that the performance on multiclass quantification is inferior to the results
obtained in the binary setting. Our results can guide practitioners who intend
to apply quantification algorithms and help researchers to identify
opportunities for future research.
