Training general representations for remote sensing using in-domain
  knowledge
  Automatically finding good and general remote sensing representations allows
to perform transfer learning on a wide range of applications - improving the
accuracy and reducing the required number of training samples. This paper
investigates development of generic remote sensing representations, and
explores which characteristics are important for a dataset to be a good source
for representation learning. For this analysis, five diverse remote sensing
datasets are selected and used for both, disjoint upstream representation
learning and downstream model training and evaluation. A common evaluation
protocol is used to establish baselines for these datasets that achieve
state-of-the-art performance. As the results indicate, especially with a low
number of available training samples a significant performance enhancement can
be observed when including additionally in-domain data in comparison to
training models from scratch or fine-tuning only on ImageNet (up to 11% and
40%, respectively, at 100 training samples). All datasets and pretrained
representation models are published online.
