Bayesian Meta-Learning Through Variational Gaussian Processes
  Recent advances in the field of meta-learning have tackled domains consisting
of large numbers of small ("few-shot") supervised learning tasks. Meta-learning
algorithms must be able to rapidly adapt to any individual few-shot task,
fitting to a small support set within a task and using it to predict the labels
of the task's query set. This problem setting can be extended to the Bayesian
context, wherein rather than predicting a single label for each query data
point, a model predicts a distribution of labels capturing its uncertainty.
Successful methods in this domain include Bayesian ensembling of MAML-based
models, Bayesian neural networks, and Gaussian processes with learned deep
kernel and mean functions. While Gaussian processes have a robust Bayesian
interpretation in the meta-learning context, they do not naturally model
non-Gaussian predictive posteriors for expressing uncertainty. In this paper,
we design a theoretically principled method, VMGP, extending
Gaussian-process-based meta-learning to allow for high-quality, arbitrary
non-Gaussian uncertainty predictions. On benchmark environments with complex
non-smooth or discontinuous structure, we find our VMGP method performs
significantly better than existing Bayesian meta-learning baselines.
