The Utility of Feature Reuse: Transfer Learning in Data-Starved Regimes
  The use of transfer learning with deep neural networks has increasingly
become widespread for deploying well-tested computer vision systems to newer
domains, especially those with limited datasets. We describe a transfer
learning use case for a domain with a data-starved regime, having fewer than
100 labeled target samples. We evaluate the effectiveness of convolutional
feature extraction and fine-tuning of overparameterized models with respect to
the size of target training data, as well as their generalization performance
on data with covariate shift, or out-of-distribution (OOD) data. Our
experiments show that both overparameterization and feature reuse contribute to
successful application of transfer learning in training image classifiers in
data-starved regimes.
