Letters of the Alphabet: Discovering Natural Feature Sets
  Deep learning networks find intricate features in large datasets using the
backpropagation algorithm. This algorithm repeatedly adjusts the network
connections.' weights and examining the "hidden" nodes behavior between the
input and output layer provides better insight into how neural networks create
feature representations. Experiments built on each other show that activity
differences computed within a layer can guide learning. A simple neural network
is used, which includes a data set comprised of the alphabet letters, where
each letter forms 81 input nodes comprised of 0 and 1s and a single hidden
layer and an output layer. The first experiment explains how the hidden layers
in this simple neural network represent the input data's features. The second
experiment attempts to reverse-engineer the neural network to find the
alphabet's natural feature sets. As the network interprets features, we can
understand how it derives the natural feature sets for a given data. This
understanding is essential to delve deeper into deep generative models, such as
Boltzmann machines. Deep generative models are a class of unsupervised deep
learning algorithms. The primary function of deep generative models is to find
the natural feature sets for a given data set.
