Less-forgetting Learning in Deep Neural Networks
  A catastrophic forgetting problem makes deep neural networks forget the
previously learned information, when learning data collected in new
environments, such as by different sensors or in different light conditions.
This paper presents a new method for alleviating the catastrophic forgetting
problem. Unlike previous research, our method does not use any information from
the source domain. Surprisingly, our method is very effective to forget less of
the information in the source domain, and we show the effectiveness of our
method using several experiments. Furthermore, we observed that the forgetting
problem occurs between mini-batches when performing general training processes
using stochastic gradient descent methods, and this problem is one of the
factors that degrades generalization performance of the network. We also try to
solve this problem using the proposed method. Finally, we show our
less-forgetting learning method is also helpful to improve the performance of
deep neural networks in terms of recognition rates.
