A New Fuzzy Stacked Generalization Technique and Analysis of its
  Performance
  In this study, a new Stacked Generalization technique called Fuzzy Stacked
Generalization (FSG) is proposed to minimize the difference between N -sample
and large-sample classification error of the Nearest Neighbor classifier. The
proposed FSG employs a new hierarchical distance learning strategy to minimize
the error difference. For this purpose, we first construct an ensemble of
base-layer fuzzy k- Nearest Neighbor (k-NN) classifiers, each of which receives
a different feature set extracted from the same sample set. The fuzzy
membership values computed at the decision space of each fuzzy k-NN classifier
are concatenated to form the feature vectors of a fusion space. Finally, the
feature vectors are fed to a meta-layer classifier to learn the degree of
accuracy of the decisions of the base-layer classifiers for meta-layer
classification. Rather than the power of the individual base layer-classifiers,
diversity and cooperation of the classifiers become an important issue to
improve the overall performance of the proposed FSG. A weak base-layer
classifier may boost the overall performance more than a strong classifier, if
it is capable of recognizing the samples, which are not recognized by the rest
of the classifiers, in its own feature space. The experiments explore the type
of the collaboration among the individual classifiers required for an improved
performance of the suggested architecture. Experiments on multiple feature
real-world datasets show that the proposed FSG performs better than the state
of the art ensemble learning algorithms such as Adaboost, Random Subspace and
Rotation Forest. On the other hand, compatible performances are observed in the
experiments on single feature multi-attribute datasets.
