Meta-Learning with Network Pruning
  Meta-learning is a powerful paradigm for few-shot learning. Although with
remarkable success witnessed in many applications, the existing optimization
based meta-learning models with over-parameterized neural networks have been
evidenced to ovetfit on training tasks. To remedy this deficiency, we propose a
network pruning based meta-learning approach for overfitting reduction via
explicitly controlling the capacity of network. A uniform concentration
analysis reveals the benefit of network capacity constraint for reducing
generalization gap of the proposed meta-learner. We have implemented our
approach on top of Reptile assembled with two network pruning routines:
Dense-Sparse-Dense (DSD) and Iterative Hard Thresholding (IHT). Extensive
experimental results on benchmark datasets with different over-parameterized
deep networks demonstrate that our method not only effectively alleviates
meta-overfitting but also in many cases improves the overall generalization
performance when applied to few-shot classification tasks.
