Cross-domain Few-shot Meta-learning Using Stacking
Cross-domain few-shot meta-learning (CDFSML) addresses learning problems
where knowledge needs to be transferred from several source domains into an
instance-scarce target domain with an explicitly different input distribution.
Recently published CDFSML methods generally construct a "universal model" that
combines knowledge of multiple source domains into one backbone feature
extractor. This enables efficient inference but necessitates re-computation of
the backbone whenever a new source domain is added. Moreover, state-of-the-art
methods derive their universal model from a collection of backbones -- normally
one for each source domain -- and the backbones may be constrained to have the
same architecture as the universal model. We propose a CDFSML method that is
inspired by the classic stacking approach to meta learning. It imposes no
constraints on the backbones' architecture or feature shape and does not incur
the computational overhead of (re-)computing a universal model. Given a
target-domain task, it fine-tunes each backbone independently, uses
cross-validation to extract meta training data from the task's instance-scarce
support set, and learns a simple linear meta classifier from this data. We
evaluate our stacking approach on the well-known Meta-Dataset benchmark,
targeting image classification with convolutional neural networks, and show
that it often yields substantially higher accuracy than competing methods.