Automatic Recall Machines: Internal Replay, Continual Learning and the
  Brain
  Replay in neural networks involves training on sequential data with memorized
samples, which counteracts forgetting of previous behavior caused by
non-stationarity. We present a method where these auxiliary samples are
generated on the fly, given only the model that is being trained for the
assessed objective, without extraneous buffers or generator networks. Instead
the implicit memory of learned samples within the assessed model itself is
exploited. Furthermore, whereas existing work focuses on reinforcing the full
seen data distribution, we show that optimizing for not forgetting calls for
the generation of samples that are specialized to each real training batch,
which is more efficient and scalable. We consider high-level parallels with the
brain, notably the use of a single model for inference and recall, the
dependency of recalled samples on the current environment batch, top-down
modulation of activations and learning, abstract recall, and the dependency
between the degree to which a task is learned and the degree to which it is
recalled. These characteristics emerge naturally from the method without being
controlled for.
