Information-theoretic generalization bounds for black-box learning
  algorithms
  We derive information-theoretic generalization bounds for supervised learning
algorithms based on the information contained in predictions rather than in the
output of the training algorithm. These bounds improve over the existing
information-theoretic bounds, are applicable to a wider range of algorithms,
and solve two key challenges: (a) they give meaningful results for
deterministic algorithms and (b) they are significantly easier to estimate. We
show experimentally that the proposed bounds closely follow the generalization
gap in practical scenarios for deep learning.
