Capsule Network Performance on Complex Data
  In recent years, convolutional neural networks (CNN) have played an important
role in the field of deep learning. Variants of CNN's have proven to be very
successful in classification tasks across different domains. However, there are
two big drawbacks to CNN's: their failure to take into account of important
spatial hierarchies between features, and their lack of rotational invariance.
As long as certain key features of an object are present in the test data,
CNN's classify the test data as the object, disregarding features' relative
spatial orientation to each other. This causes false positives. The lack of
rotational invariance in CNN's would cause the network to incorrectly assign
the object another label, causing false negatives. To address this concern,
Hinton et al. propose a novel type of neural network using the concept of
capsules in a recent paper. With the use of dynamic routing and reconstruction
regularization, the capsule network model would be both rotation invariant and
spatially aware. The capsule network has shown its potential by achieving a
state-of-the-art result of 0.25% test error on MNIST without data augmentation
such as rotation and scaling, better than the previous baseline of 0.39%. To
further test out the application of capsule networks on data with higher
dimensionality, we attempt to find the best set of configurations that yield
the optimal test error on CIFAR10 dataset.
