Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform
  Stability
  We are motivated by the problem of providing strong generalization guarantees
in the context of meta-learning. Existing generalization bounds are either
challenging to evaluate or provide vacuous guarantees in even relatively simple
settings. We derive a probably approximately correct (PAC) bound for
gradient-based meta-learning using two different generalization frameworks in
order to deal with the qualitatively different challenges of generalization at
the "base" and "meta" levels. We employ bounds for uniformly stable algorithms
at the base level and bounds from the PAC-Bayes framework at the meta level.
The result of this approach is a novel PAC bound that is tighter when the base
learner adapts quickly, which is precisely the goal of meta-learning. We show
that our bound provides a tighter guarantee than other bounds on a toy
non-convex problem on the unit sphere and a text-based classification example.
We also present a practical regularization scheme motivated by the bound in
settings where the bound is loose and demonstrate improved performance over
baseline techniques.
