Learning to Remember: A Synaptic Plasticity Driven Framework for
  Continual Learning
  Models trained in the context of continual learning (CL) should be able to
learn from a stream of data over an undefined period of time. The main
challenges herein are: 1) maintaining old knowledge while simultaneously
benefiting from it when learning new tasks, and 2) guaranteeing model
scalability with a growing amount of data to learn from. In order to tackle
these challenges, we introduce Dynamic Generative Memory (DGM) - a synaptic
plasticity driven framework for continual learning. DGM relies on conditional
generative adversarial networks with learnable connection plasticity realized
with neural masking. Specifically, we evaluate two variants of neural masking:
applied to (i) layer activations and (ii) to connection weights directly.
Furthermore, we propose a dynamic network expansion mechanism that ensures
sufficient model capacity to accommodate for continually incoming tasks. The
amount of added capacity is determined dynamically from the learned binary
mask. We evaluate DGM in the continual class-incremental setup on visual
classification tasks.
