An Outlier Detection-based Tree Selection Approach to Extreme Pruning of
  Random Forests
  Random Forest (RF) is an ensemble classification technique that was developed
by Breiman over a decade ago. Compared with other ensemble techniques, it has
proved its accuracy and superiority. Many researchers, however, believe that
there is still room for enhancing and improving its performance in terms of
predictive accuracy. This explains why, over the past decade, there have been
many extensions of RF where each extension employed a variety of techniques and
strategies to improve certain aspect(s) of RF. Since it has been proven
empirically that ensembles tend to yield better results when there is a
significant diversity among the constituent models, the objective of this paper
is twofolds. First, it investigates how an unsupervised learning technique,
namely, Local Outlier Factor (LOF) can be used to identify diverse trees in the
RF. Second, trees with the highest LOF scores are then used to produce an
extension of RF termed LOFB-DRF that is much smaller in size than RF, and yet
performs at least as good as RF, but mostly exhibits higher performance in
terms of accuracy. The latter refers to a known technique called ensemble
pruning. Experimental results on 10 real datasets prove the superiority of our
proposed extension over the traditional RF. Unprecedented pruning levels
reaching 99% have been achieved at the time of boosting the predictive accuracy
of the ensemble. The notably high pruning level makes the technique a good
candidate for real-time applications.
