Ensemble pruning via an integer programming approach with diversity constraints
Ensemble learning combines multiple classifiers in the hope of obtaining
better predictive performance. Empirical studies have shown that ensemble
pruning, that is, choosing an appropriate subset of the available classifiers,
can lead to comparable or better predictions than using all classifiers. In
this paper, we consider a binary classification problem and propose an integer
programming (IP) approach for selecting optimal classifier subsets. We propose
a flexible objective function to adapt to desired criteria of different
datasets. We also propose constraints to ensure minimum diversity levels in the
ensemble. Despite the general case of IP being NP-Hard, state-of-the-art
solvers are able to quickly obtain good solutions for datasets with up to 60000
data points. Our approach yields competitive results when compared to some of
the best and most used pruning methods in literature.