An empirical study of pretrained representations for few-shot
  classification
  Recent algorithms with state-of-the-art few-shot classification results start
their procedure by computing data features output by a large pretrained model.
In this paper we systematically investigate which models provide the best
representations for a few-shot image classification task when pretrained on the
Imagenet dataset. We test their representations when used as the starting point
for different few-shot classification algorithms. We observe that models
trained on a supervised classification task have higher performance than models
trained in an unsupervised manner even when transferred to out-of-distribution
datasets. Models trained with adversarial robustness transfer better, while
having slightly lower accuracy than supervised models.
