A Framework of Meta Functional Learning for Regularising Knowledge
  Transfer
  Machine learning classifiers' capability is largely dependent on the scale of
available training data and limited by the model overfitting in data-scarce
learning tasks. To address this problem, this work proposes a novel framework
of Meta Functional Learning (MFL) by meta-learning a generalisable functional
model from data-rich tasks whilst simultaneously regularising knowledge
transfer to data-scarce tasks. The MFL computes meta-knowledge on functional
regularisation generalisable to different learning tasks by which functional
training on limited labelled data promotes more discriminative functions to be
learned. Based on this framework, we formulate three variants of MFL: MFL with
Prototypes (MFL-P) which learns a functional by auxiliary prototypes, Composite
MFL (ComMFL) that transfers knowledge from both functional space and
representational space, and MFL with Iterative Updates (MFL-IU) which improves
knowledge transfer regularisation from MFL by progressively learning the
functional regularisation in knowledge transfer. Moreover, we generalise these
variants for knowledge transfer regularisation from binary classifiers to
multi-class classifiers. Extensive experiments on two few-shot learning
scenarios, Few-Shot Learning (FSL) and Cross-Domain Few-Shot Learning (CD-FSL),
show that meta functional learning for knowledge transfer regularisation can
improve FSL classifiers.
