Bayesian Few-Shot Classification with One-vs-Each P\'olya-Gamma
  Augmented Gaussian Processes
  Few-shot classification (FSC), the task of adapting a classifier to unseen
classes given a small labeled dataset, is an important step on the path toward
human-like machine learning. Bayesian methods are well-suited to tackling the
fundamental issue of overfitting in the few-shot scenario because they allow
practitioners to specify prior beliefs and update those beliefs in light of
observed data. Contemporary approaches to Bayesian few-shot classification
maintain a posterior distribution over model parameters, which is slow and
requires storage that scales with model size. Instead, we propose a Gaussian
process classifier based on a novel combination of P\'olya-Gamma augmentation
and the one-vs-each softmax approximation that allows us to efficiently
marginalize over functions rather than model parameters. We demonstrate
improved accuracy and uncertainty quantification on both standard few-shot
classification benchmarks and few-shot domain transfer tasks.
