Policy Consolidation for Continual Reinforcement Learning
  We propose a method for tackling catastrophic forgetting in deep
reinforcement learning that is \textit{agnostic} to the timescale of changes in
the distribution of experiences, does not require knowledge of task boundaries,
and can adapt in \textit{continuously} changing environments. In our
\textit{policy consolidation} model, the policy network interacts with a
cascade of hidden networks that simultaneously remember the agent's policy at a
range of timescales and regularise the current policy by its own history,
thereby improving its ability to learn without forgetting. We find that the
model improves continual learning relative to baselines on a number of
continuous control tasks in single-task, alternating two-task, and multi-agent
competitive self-play settings.
