On the Current State of Research in Explaining Ensemble Performance
  Using Margins
  Empirical evidence shows that ensembles, such as bagging, boosting, random
and rotation forests, generally perform better in terms of their generalization
error than individual classifiers. To explain this performance, Schapire et al.
(1998) developed an upper bound on the generalization error of an ensemble
based on the margins of the training data, from which it was concluded that
larger margins should lead to lower generalization error, everything else being
equal. Many other researchers have backed this assumption and presented tighter
bounds on the generalization error based on either the margins or functions of
the margins. For instance, Shen and Li (2010) provide evidence suggesting that
the generalization error of a voting classifier might be reduced by increasing
the mean and decreasing the variance of the margins. In this article we propose
several techniques and empirically test whether the current state of research
in explaining ensemble performance holds. We evaluate the proposed methods
through experiments with real and simulated data sets.
