Geometry and Generalization: Eigenvalues as predictors of where a
  network will fail to generalize
  We study the deformation of the input space by a trained autoencoder via the
Jacobians of the trained weight matrices. In doing so, we prove bounds for the
mean squared errors for points in the input space, under assumptions regarding
the orthogonality of the eigenvectors. We also show that the trace and the
product of the eigenvalues of the Jacobian matrices is a good predictor of the
MSE on test points. This is a dataset independent means of testing an
autoencoder's ability to generalize on new input. Namely, no knowledge of the
dataset on which the network was trained is needed, only the parameters of the
trained model.
