Quantised Transforming Auto-Encoders: Achieving Equivariance to
  Arbitrary Transformations in Deep Networks
  In this work we investigate how to achieve equivariance to input
transformations in deep networks, purely from data, without being given a model
of those transformations. Convolutional Neural Networks (CNNs), for example,
are equivariant to image translation, a transformation that can be easily
modelled (by shifting the pixels vertically or horizontally). Other
transformations, such as out-of-plane rotations, do not admit a simple analytic
model. We propose an auto-encoder architecture whose embedding obeys an
arbitrary set of equivariance relations simultaneously, such as translation,
rotation, colour changes, and many others. This means that it can take an input
image, and produce versions transformed by a given amount that were not
observed before (e.g. a different point of view of the same object, or a colour
variation). Despite extending to many (even non-geometric) transformations, our
model reduces exactly to a CNN in the special case of translation-equivariance.
Equivariances are important for the interpretability and robustness of deep
networks, and we demonstrate results of successful re-rendering of transformed
versions of input images on several synthetic and real datasets, as well as
results on object pose estimation.
