Examining the Use of Neural Networks for Feature Extraction: A
  Comparative Analysis using Deep Learning, Support Vector Machines, and
  K-Nearest Neighbor Classifiers
  Neural networks in many varieties are touted as very powerful machine
learning tools because of their ability to distill large amounts of information
from different forms of data, extracting complex features and enabling powerful
classification abilities. In this study, we use neural networks to extract
features from both images and numeric data and use these extracted features as
inputs for other machine learning models, namely support vector machines (SVMs)
and k-nearest neighbor classifiers (KNNs), in order to see if
neural-network-extracted features enhance the capabilities of these models. We
tested 7 different neural network architectures in this manner, 4 for images
and 3 for numeric data, training each for varying lengths of time and then
comparing the results of the neural network independently to those of an SVM
and KNN on the data, and finally comparing these results to models of SVM and
KNN trained using features extracted via the neural network architecture. This
process was repeated on 3 different image datasets and 2 different numeric
datasets. The results show that, in many cases, the features extracted using
the neural network significantly improve the capabilities of SVMs and KNNs
compared to running these algorithms on the raw features, and in some cases
also surpass the performance of the neural network alone. This in turn suggests
that it may be a reasonable practice to use neural networks as a means to
extract features for classification by other machine learning models for some
datasets.
