Tracking translation invariance in CNNs
  Although Convolutional Neural Networks (CNNs) are widely used, their
translation invariance (ability to deal with translated inputs) is still
subject to some controversy. We explore this question using
translation-sensitivity maps to quantify how sensitive a standard CNN is to a
translated input. We propose the use of Cosine Similarity as sensitivity metric
over Euclidean Distance, and discuss the importance of restricting the
dimensionality of either of these metrics when comparing architectures. Our
main focus is to investigate the effect of different architectural components
of a standard CNN on that network's sensitivity to translation. By varying
convolutional kernel sizes and amounts of zero padding, we control the size of
the feature maps produced, allowing us to quantify the extent to which these
elements influence translation invariance. We also measure translation
invariance at different locations within the CNN to determine the extent to
which convolutional and fully connected layers, respectively, contribute to the
translation invariance of a CNN as a whole. Our analysis indicates that both
convolutional kernel size and feature map size have a systematic influence on
translation invariance. We also see that convolutional layers contribute less
than expected to translation invariance, when not specifically forced to do so.
