Label Mapping Neural Networks with Response Consolidation for Class
  Incremental Learning
  Class incremental learning refers to a special multi-class classification
task, in which the number of classes is not fixed but is increasing with the
continual arrival of new data. Existing researches mainly focused on solving
catastrophic forgetting problem in class incremental learning. To this end,
however, these models still require the old classes cached in the auxiliary
data structure or models, which is inefficient in space or time. In this paper,
it is the first time to discuss the difficulty without support of old classes
in class incremental learning, which is called as softmax suppression problem.
To address these challenges, we develop a new model named Label Mapping with
Response Consolidation (LMRC), which need not access the old classes anymore.
We propose the Label Mapping algorithm combined with the multi-head neural
network for mitigating the softmax suppression problem, and propose the
Response Consolidation method to overcome the catastrophic forgetting problem.
Experimental results on the benchmark datasets show that our proposed method
achieves much better performance compared to the related methods in different
scenarios.
