A Survey of Techniques All Classifiers Can Learn from Deep Networks:
  Models, Optimizations, and Regularization
  Deep neural networks have introduced novel and useful tools to the machine
learning community. Other types of classifiers can potentially make use of
these tools as well to improve their performance and generality. This paper
reviews the current state of the art for deep learning classifier technologies
that are being used outside of deep neural networks. Non-network classifiers
can employ many components found in deep neural network architectures. In this
paper, we review the feature learning, optimization, and regularization methods
that form a core of deep network technologies. We then survey non-neural
network learning algorithms that make innovative use of these methods to
improve classification. Because many opportunities and challenges still exist,
we discuss directions that can be pursued to expand the area of deep learning
for a variety of classification algorithms.
