Pseudo-Rehearsal: Achieving Deep Reinforcement Learning without
  Catastrophic Forgetting
  Neural networks can achieve excellent results in a wide variety of
applications. However, when they attempt to sequentially learn, they tend to
learn the new task while catastrophically forgetting previous ones. We propose
a model that overcomes catastrophic forgetting in sequential reinforcement
learning by combining ideas from continual learning in both the image
classification domain and the reinforcement learning domain. This model
features a dual memory system which separates continual learning from
reinforcement learning and a pseudo-rehearsal system that "recalls" items
representative of previous tasks via a deep generative network. Our model
sequentially learns Atari 2600 games without demonstrating catastrophic
forgetting and continues to perform above human level on all three games. This
result is achieved without: demanding additional storage requirements as the
number of tasks increases, storing raw data or revisiting past tasks. In
comparison, previous state-of-the-art solutions are substantially more
vulnerable to forgetting on these complex deep reinforcement learning tasks.
