Improving Content-Invariance in Gated Autoencoders for 2D and 3D Object
  Rotation
  Content-invariance in mapping codes learned by GAEs is a useful feature for
various relation learning tasks. In this paper we show that the
content-invariance of mapping codes for images of 2D and 3D rotated objects can
be substantially improved by extending the standard GAE loss (symmetric
reconstruction error) with a regularization term that penalizes the symmetric
cross-reconstruction error. This error term involves reconstruction of pairs
with mapping codes obtained from other pairs exhibiting similar
transformations. Although this would principally require knowledge of the
transformations exhibited by training pairs, our experiments show that a
bootstrapping approach can sidestep this issue, and that the regularization
term can effectively be used in an unsupervised setting.
