Learning Bayesian Sparse Networks with Full Experience Replay for
  Continual Learning
  Continual Learning (CL) methods aim to enable machine learning models to
learn new tasks without catastrophic forgetting of those that have been
previously mastered. Existing CL approaches often keep a buffer of
previously-seen samples, perform knowledge distillation, or use regularization
techniques towards this goal. Despite their performance, they still suffer from
interference across tasks which leads to catastrophic forgetting. To ameliorate
this problem, we propose to only activate and select sparse neurons for
learning current and past tasks at any stage. More parameters space and model
capacity can thus be reserved for the future tasks. This minimizes the
interference between parameters for different tasks. To do so, we propose a
Sparse neural Network for Continual Learning (SNCL), which employs variational
Bayesian sparsity priors on the activations of the neurons in all layers. Full
Experience Replay (FER) provides effective supervision in learning the sparse
activations of the neurons in different layers. A loss-aware reservoir-sampling
strategy is developed to maintain the memory buffer. The proposed method is
agnostic as to the network structures and the task boundaries. Experiments on
different datasets show that our approach achieves state-of-the-art performance
for mitigating forgetting.
