Learning to Remember, Forget and Ignore using Attention Control in
  Memory
  Typical neural networks with external memory do not effectively separate
capacity for episodic and working memory as is required for reasoning in
humans. Applying knowledge gained from psychological studies, we designed a new
model called Differentiable Working Memory (DWM) in order to specifically
emulate human working memory. As it shows the same functional characteristics
as working memory, it robustly learns psychology inspired tasks and converges
faster than comparable state-of-the-art models. Moreover, the DWM model
successfully generalizes to sequences two orders of magnitude longer than the
ones used in training. Our in-depth analysis shows that the behavior of DWM is
interpretable and that it learns to have fine control over memory, allowing it
to retain, ignore or forget information based on its relevance.
