Empirical Bayes Transductive Meta-Learning with Synthetic Gradients
  We propose a meta-learning approach that learns from multiple tasks in a
transductive setting, by leveraging the unlabeled query set in addition to the
support set to generate a more powerful model for each task. To develop our
framework, we revisit the empirical Bayes formulation for multi-task learning.
The evidence lower bound of the marginal log-likelihood of empirical Bayes
decomposes as a sum of local KL divergences between the variational posterior
and the true posterior on the query set of each task. We derive a novel
amortized variational inference that couples all the variational posteriors via
a meta-model, which consists of a synthetic gradient network and an
initialization network. Each variational posterior is derived from synthetic
gradient descent to approximate the true posterior on the query set, although
where we do not have access to the true gradient. Our results on the
Mini-ImageNet and CIFAR-FS benchmarks for episodic few-shot classification
outperform previous state-of-the-art methods. Besides, we conduct two zero-shot
learning experiments to further explore the potential of the synthetic
gradient.
