Building Deep, Equivariant Capsule Networks
  Capsule networks are constrained by the parameter-expensive nature of their
layers, and the general lack of provable equivariance guarantees. We present a
variation of capsule networks that aims to remedy this. We identify that
learning all pair-wise part-whole relationships between capsules of successive
layers is inefficient. Further, we also realise that the choice of prediction
networks and the routing mechanism are both key to equivariance. Based on
these, we propose an alternative framework for capsule networks that learns to
projectively encode the manifold of pose-variations, termed the
space-of-variation (SOV), for every capsule-type of each layer. This is done
using a trainable, equivariant function defined over a grid of
group-transformations. Thus, the prediction-phase of routing involves
projection into the SOV of a deeper capsule using the corresponding function.
As a specific instantiation of this idea, and also in order to reap the
benefits of increased parameter-sharing, we use type-homogeneous
group-equivariant convolutions of shallower capsules in this phase. We also
introduce an equivariant routing mechanism based on degree-centrality. We show
that this particular instance of our general model is equivariant, and hence
preserves the compositional representation of an input under transformations.
We conduct several experiments on standard object-classification datasets that
showcase the increased transformation-robustness, as well as general
performance, of our model to several capsule baselines.
