Eigenvalues of Autoencoders in Training and at Initialization
  In this paper, we investigate the evolution of autoencoders near their
initialization. In particular, we study the distribution of the eigenvalues of
the Jacobian matrices of autoencoders early in the training process, training
on the MNIST data set. We find that autoencoders that have not been trained
have eigenvalue distributions that are qualitatively different from those which
have been trained for a long time ($>$100 epochs). Additionally, we find that
even at early epochs, these eigenvalue distributions rapidly become
qualitatively similar to those of the fully trained autoencoders. We also
compare the eigenvalues at initialization to pertinent theoretical work on the
eigenvalues of random matrices and the products of such matrices.
