Training capsules as a routing-weighted product of expert neurons
  Capsules are the multidimensional analogue to scalar neurons in neural
networks, and because they are multidimensional, much more complex routing
schemes can be used to pass information forward through the network than what
can be used in traditional neural networks. This work treats capsules as
collections of neurons in a fully connected neural network, where sub-networks
connecting capsules are weighted according to the routing coefficients
determined by routing by agreement. An energy function is designed to reflect
this model, and it follows that capsule networks with dynamic routing can be
formulated as a product of expert neurons. By alternating between dynamic
routing, which acts to both find subnetworks within the overall network as well
as to mix the model distribution, and updating the parameters by the gradient
of the contrastive divergence, a bottom-up, unsupervised learning algorithm is
constructed for capsule networks with dynamic routing. The model and its
training algorithm are qualitatively tested in the generative sense, and is
able to produce realistic looking images from standard vision datasets.
