Continual Learning with Bayesian Model based on a Fixed Pre-trained
  Feature Extractor
  Deep learning has shown its human-level performance in various applications.
However, current deep learning models are characterised by catastrophic
forgetting of old knowledge when learning new classes. This poses a challenge
particularly in intelligent diagnosis systems where initially only training
data of a limited number of diseases are available. In this case, updating the
intelligent system with data of new diseases would inevitably downgrade its
performance on previously learned diseases. Inspired by the process of learning
new knowledge in human brains, we propose a Bayesian generative model for
continual learning built on a fixed pre-trained feature extractor. In this
model, knowledge of each old class can be compactly represented by a collection
of statistical distributions, e.g. with Gaussian mixture models, and naturally
kept from forgetting in continual learning over time. Unlike existing
class-incremental learning methods, the proposed approach is not sensitive to
the continual learning process and can be additionally well applied to the
data-incremental learning scenario. Experiments on multiple medical and natural
image classification tasks showed that the proposed approach outperforms
state-of-the-art approaches which even keep some images of old classes during
continual learning of new classes.
