Information Theoretic Meta Learning with Gaussian Processes
  We formulate meta learning using information theoretic concepts; namely,
mutual information and the information bottleneck. The idea is to learn a
stochastic representation or encoding of the task description, given by a
training set, that is highly informative about predicting the validation set.
By making use of variational approximations to the mutual information, we
derive a general and tractable framework for meta learning. This framework
unifies existing gradient-based algorithms and also allows us to derive new
algorithms. In particular, we develop a memory-based algorithm that uses
Gaussian processes to obtain non-parametric encoding representations. We
demonstrate our method on a few-shot regression problem and on four few-shot
classification problems, obtaining competitive accuracy when compared to
existing baselines.
