Less Is More: A Comprehensive Framework for the Number of Components of
  Ensemble Classifiers
  The number of component classifiers chosen for an ensemble greatly impacts
the prediction ability. In this paper, we use a geometric framework for a
priori determining the ensemble size, which is applicable to most of existing
batch and online ensemble classifiers. There are only a limited number of
studies on the ensemble size examining Majority Voting (MV) and Weighted
Majority Voting (WMV). Almost all of them are designed for batch-mode, hardly
addressing online environments. Big data dimensions and resource limitations,
in terms of time and memory, make determination of ensemble size crucial,
especially for online environments. For the MV aggregation rule, our framework
proves that the more strong components we add to the ensemble, the more
accurate predictions we can achieve. For the WMV aggregation rule, our
framework proves the existence of an ideal number of components, which is equal
to the number of class labels, with the premise that components are completely
independent of each other and strong enough. While giving the exact definition
for a strong and independent classifier in the context of an ensemble is a
challenging task, our proposed geometric framework provides a theoretical
explanation of diversity and its impact on the accuracy of predictions. We
conduct a series of experimental evaluations to show the practical value of our
theorems and existing challenges.
