Capsule Routing via Variational Bayes
  Capsule networks are a recently proposed type of neural network shown to
outperform alternatives in challenging shape recognition tasks. In capsule
networks, scalar neurons are replaced with capsule vectors or matrices, whose
entries represent different properties of objects. The relationships between
objects and their parts are learned via trainable viewpoint-invariant
transformation matrices, and the presence of a given object is decided by the
level of agreement among votes from its parts. This interaction occurs between
capsule layers and is a process called routing-by-agreement. In this paper, we
propose a new capsule routing algorithm derived from Variational Bayes for
fitting a mixture of transforming gaussians, and show it is possible transform
our capsule network into a Capsule-VAE. Our Bayesian approach addresses some of
the inherent weaknesses of MLE based models such as the variance-collapse by
modelling uncertainty over capsule pose parameters. We outperform the
state-of-the-art on smallNORB using 50% fewer capsules than previously
reported, achieve competitive performances on CIFAR-10, Fashion-MNIST, SVHN,
and demonstrate significant improvement in MNIST to affNIST generalisation over
previous works.
