Deep Super Learner: A Deep Ensemble for Classification Problems
  Deep learning has become very popular for tasks such as predictive modeling
and pattern recognition in handling big data. Deep learning is a powerful
machine learning method that extracts lower level features and feeds them
forward for the next layer to identify higher level features that improve
performance. However, deep neural networks have drawbacks, which include many
hyper-parameters and infinite architectures, opaqueness into results, and
relatively slower convergence on smaller datasets. While traditional machine
learning algorithms can address these drawbacks, they are not typically capable
of the performance levels achieved by deep neural networks. To improve
performance, ensemble methods are used to combine multiple base learners. Super
learning is an ensemble that finds the optimal combination of diverse learning
algorithms. This paper proposes deep super learning as an approach which
achieves log loss and accuracy results competitive to deep neural networks
while employing traditional machine learning algorithms in a hierarchical
structure. The deep super learner is flexible, adaptable, and easy to train
with good performance across different tasks using identical hyper-parameter
values. Using traditional machine learning requires fewer hyper-parameters,
allows transparency into results, and has relatively fast convergence on
smaller datasets. Experimental results show that the deep super learner has
superior performance compared to the individual base learners, single-layer
ensembles, and in some cases deep neural networks. Performance of the deep
super learner may further be improved with task-specific tuning.
