Incremental Classifier Learning Based on PEDCC-Loss and Cosine Distance
  The main purpose of incremental learning is to learn new knowledge while not
forgetting the knowledge which have been learned before. At present, the main
challenge in this area is the catastrophe forgetting, namely the network will
lose their performance in the old tasks after training for new tasks. In this
paper, we introduce an ensemble method of incremental classifier to alleviate
this problem, which is based on the cosine distance between the output feature
and the pre-defined center, and can let each task to be preserved in different
networks. During training, we make use of PEDCC-Loss to train the CNN network.
In the stage of testing, the prediction is determined by the cosine distance
between the network latent features and pre-defined center. The experimental
results on EMINST and CIFAR100 show that our method outperforms the recent LwF
method, which use the knowledge distillation, and iCaRL method, which keep some
old samples while training for new task. The method can achieve the goal of not
forgetting old knowledge while training new classes, and solve the problem of
catastrophic forgetting better.
