Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks
  New classes arise frequently in our ever-changing world, e.g., emerging
topics in social media and new types of products in e-commerce. A model should
recognize new classes and meanwhile maintain discriminability over old classes.
Under severe circumstances, only limited novel instances are available to
incrementally update the model. The task of recognizing few-shot new classes
without forgetting old classes is called few-shot class-incremental learning
(FSCIL). In this work, we propose a new paradigm for FSCIL based on
meta-learning by LearnIng Multi-phase Incremental Tasks (LIMIT), which
synthesizes fake FSCIL tasks from the base dataset. The data format of fake
tasks is consistent with the `real' incremental tasks, and we can build a
generalizable feature space for the unseen tasks through meta-learning.
Besides, LIMIT also constructs a calibration module based on transformer, which
calibrates the old class classifiers and new class prototypes into the same
scale and fills in the semantic gap. The calibration module also adaptively
contextualizes the instance-specific embedding with a set-to-set function.
LIMIT efficiently adapts to new classes and meanwhile resists forgetting over
old classes. Experiments on three benchmark datasets (CIFAR100, miniImageNet,
and CUB200) and large-scale dataset, i.e., ImageNet ILSVRC2012 validate that
LIMIT achieves state-of-the-art performance.
