Episodic memory for continual model learning
  Both the human brain and artificial learning agents operating in real-world
or comparably complex environments are faced with the challenge of online model
selection. In principle this challenge can be overcome: hierarchical Bayesian
inference provides a principled method for model selection and it converges on
the same posterior for both off-line (i.e. batch) and online learning. However,
maintaining a parameter posterior for each model in parallel has in general an
even higher memory cost than storing the entire data set and is consequently
clearly unfeasible. Alternatively, maintaining only a limited set of models in
memory could limit memory requirements. However, sufficient statistics for one
model will usually be insufficient for fitting a different kind of model,
meaning that the agent loses information with each model change. We propose
that episodic memory can circumvent the challenge of limited memory-capacity
online model selection by retaining a selected subset of data points. We design
a method to compute the quantities necessary for model selection even when the
data is discarded and only statistics of one (or few) learnt models are
available. We demonstrate on a simple model that a limited-sized episodic
memory buffer, when the content is optimised to retain data with statistics not
matching the current representation, can resolve the fundamental challenge of
online model selection.
