No Routing Needed Between Capsules
  Most capsule network designs rely on traditional matrix multiplication
between capsule layers and computationally expensive routing mechanisms to deal
with the capsule dimensional entanglement that the matrix multiplication
introduces. By using Homogeneous Vector Capsules (HVCs), which use element-wise
multiplication rather than matrix multiplication, the dimensions of the
capsules remain unentangled. In this work, we study HVCs as applied to the
highly structured MNIST dataset in order to produce a direct comparison to the
capsule research direction of Geoffrey Hinton, et al. In our study, we show
that a simple convolutional neural network using HVCs performs as well as the
prior best performing capsule network on MNIST using 5.5x fewer parameters, 4x
fewer training epochs, no reconstruction sub-network, and requiring no routing
mechanism. The addition of multiple classification branches to the network
establishes a new state of the art for the MNIST dataset with an accuracy of
99.87% for an ensemble of these models, as well as establishing a new state of
the art for a single model (99.83% accurate).
