Self-Supervised Learning Aided Class-Incremental Lifelong Learning
  Lifelong or continual learning remains to be a challenge for artificial
neural network, as it is required to be both stable for preservation of old
knowledge and plastic for acquisition of new knowledge. It is common to see
previous experience get overwritten, which leads to the well-known issue of
catastrophic forgetting, especially in the scenario of class-incremental
learning (Class-IL). Recently, many lifelong learning methods have been
proposed to avoid catastrophic forgetting. However, models which learn without
replay of the input data, would encounter another problem which has been
ignored, and we refer to it as prior information loss (PIL). In training
procedure of Class-IL, as the model has no knowledge about following tasks, it
would only extract features necessary for tasks learned so far, whose
information is insufficient for joint classification. In this paper, our
empirical results on several image datasets show that PIL limits the
performance of current state-of-the-art method for Class-IL, the orthogonal
weights modification (OWM) algorithm. Furthermore, we propose to combine
self-supervised learning, which can provide effective representations without
requiring labels, with Class-IL to partly get around this problem. Experiments
show superiority of proposed method to OWM, as well as other strong baselines.
