Meta Dropout: Learning to Perturb Features for Generalization
  A machine learning model that generalizes well should obtain low errors on
unseen test examples. Thus, if we know how to optimally perturb training
examples to account for test examples, we may achieve better generalization
performance. However, obtaining such perturbation is not possible in standard
machine learning frameworks as the distribution of the test data is unknown. To
tackle this challenge, we propose a novel regularization method, meta-dropout,
which learns to perturb the latent features of training examples for
generalization in a meta-learning framework. Specifically, we meta-learn a
noise generator which outputs a multiplicative noise distribution for latent
features, to obtain low errors on the test instances in an input-dependent
manner. Then, the learned noise generator can perturb the training examples of
unseen tasks at the meta-test time for improved generalization. We validate our
method on few-shot classification datasets, whose results show that it
significantly improves the generalization performance of the base model, and
largely outperforms existing regularization methods such as information
bottleneck, manifold mixup, and information dropout.
