Dynamic Ensemble Selection VS K-NN: why and when Dynamic Selection
  obtains higher classification performance?
  Multiple classifier systems focus on the combination of classifiers to obtain
better performance than a single robust one. These systems unfold three major
phases: pool generation, selection and integration. One of the most promising
MCS approaches is Dynamic Selection (DS), which relies on finding the most
competent classifier or ensemble of classifiers to predict each test sample.
The majority of the DS techniques are based on the K-Nearest Neighbors (K-NN)
definition, and the quality of the neighborhood has a huge impact on the
performance of DS methods. In this paper, we perform an analysis comparing the
classification results of DS techniques and the K-NN classifier under different
conditions. Experiments are performed on 18 state-of-the-art DS techniques over
30 classification datasets and results show that DS methods present a
significant boost in classification accuracy even though they use the same
neighborhood as the K-NN. The reasons behind the outperformance of DS
techniques over the K-NN classifier reside in the fact that DS techniques can
deal with samples with a high degree of instance hardness (samples that are
located close to the decision border) as opposed to the K-NN. In this paper,
not only we explain why DS techniques achieve higher classification performance
than the K-NN but also when DS should be used.
