Evaluating Deep Neural Network Ensembles by Majority Voting cum
  Meta-Learning scheme
  Deep Neural Networks (DNNs) are prone to overfitting and hence have high
variance. Overfitted networks do not perform well for a new data instance. So
instead of using a single DNN as classifier we propose an ensemble of seven
independent DNN learners by varying only the input to these DNNs keeping their
architecture and intrinsic properties same. To induce variety in the training
input, for each of the seven DNNs, one-seventh of the data is deleted and
replenished by bootstrap sampling from the remaining samples. We have proposed
a novel technique for combining the prediction of the DNN learners in the
ensemble. Our method is called pre-filtering by majority voting coupled with
stacked meta-learner which performs a two-step confi-dence check for the
predictions before assigning the final class labels. All the algorithms in this
paper have been tested on five benchmark datasets name-ly, Human Activity
Recognition (HAR), Gas sensor array drift, Isolet, Spam-base and Internet
advertisements. Our ensemble approach achieves higher accuracy than a single
DNN and the average individual accuracies of DNNs in the ensemble, as well as
the baseline approaches of plurality voting and meta-learning.
