Bucketed PCA Neural Networks with Neurons Mirroring Signals
  The bucketed PCA neural network (PCA-NN) with transforms is developed here in
an effort to benchmark deep neural networks (DNN's), for problems on supervised
classification. Most classical PCA models apply PCA to the entire training data
set to establish a reductive representation and then employ non-network tools
such as high-order polynomial classifiers. In contrast, the bucketed PCA-NN
applies PCA to individual buckets which are constructed in two consecutive
phases, as well as retains a genuine architecture of a neural network. This
facilitates a fair apple-to-apple comparison to DNN's, esp. to reveal that a
major chunk of accuracy achieved by many impressive DNN's could possibly be
explained by the bucketed PCA-NN (e.g., 96% out of 98% for the MNIST data set
as an example). Compared with most DNN's, the three building blocks of the
bucketed PCA-NN are easier to comprehend conceptually - PCA, transforms, and
bucketing for error correction. Furthermore, unlike the somewhat quasi-random
neurons ubiquitously observed in DNN's, the PCA neurons resemble or mirror the
input signals and are more straightforward to decipher as a result.
