Identifying Generalization Properties in Neural Networks
  While it has not yet been proven, empirical evidence suggests that model
generalization is related to local properties of the optima which can be
described via the Hessian. We connect model generalization with the local
property of a solution under the PAC-Bayes paradigm. In particular, we prove
that model generalization ability is related to the Hessian, the higher-order
"smoothness" terms characterized by the Lipschitz constant of the Hessian, and
the scales of the parameters. Guided by the proof, we propose a metric to score
the generalization capability of the model, as well as an algorithm that
optimizes the perturbed model accordingly.
