Geometric instability of out of distribution data across autoencoder
  architecture
  We study the map learned by a family of autoencoders trained on MNIST, and
evaluated on ten different data sets created by the random selection of pixel
values according to ten different distributions. Specifically, we study the
eigenvalues of the Jacobians defined by the weight matrices of the autoencoder
at each training and evaluation point. For high enough latent dimension, we
find that each autoencoder reconstructs all the evaluation data sets as similar
\emph{generalized characters}, but that this reconstructed \emph{generalized
character} changes across autoencoder. Eigenvalue analysis shows that even when
the reconstructed image appears to be an MNIST character for all out of
distribution data sets, not all have latent representations that are close to
the latent representation of MNIST characters. All told, the eigenvalue
analysis demonstrated a great deal of geometric instability of the autoencoder
both as a function on out of distribution inputs, and across architectures on
the same set of inputs.
