Learning Compositional Structures for Deep Learning: Why
  Routing-by-agreement is Necessary
  A formal description of the compositionality of neural networks is associated
directly with the formal grammar-structure of the objects it seeks to
represent. This formal grammar-structure specifies the kind of components that
make up an object, and also the configurations they are allowed to be in. In
other words, objects can be described as a parse-tree of its components -- a
structure that can be seen as a candidate for building connection-patterns
among neurons in neural networks. We present a formal grammar description of
convolutional neural networks and capsule networks that shows how capsule
networks can enforce such parse-tree structures, while CNNs do not.
Specifically, we show that the entropy of routing coefficients in the dynamic
routing algorithm controls this ability. Thus, we introduce the entropy of
routing weights as a loss function for better compositionality among capsules.
We show by experiments, on data with a compositional structure, that the use of
this loss enables capsule networks to better detect changes in
compositionality. Our experiments show that as the entropy of the routing
weights increases, the ability to detect changes in compositionality reduces.
We see that, without routing, capsule networks perform similar to convolutional
neural networks in that both these models perform badly at detecting changes in
compositionality. Our results indicate that routing is an important part of
capsule networks -- effectively answering recent work that has questioned its
necessity. We also, by experiments on SmallNORB, CIFAR-10, and FashionMNIST,
show that this loss keeps the accuracy of capsule network models comparable to
models that do not use it .
