Reservoir Memory Machines as Neural Computers
  Differentiable neural computers extend artificial neural networks with an
explicit memory without interference, thus enabling the model to perform
classic computation tasks such as graph traversal. However, such models are
difficult to train, requiring long training times and large datasets. In this
work, we achieve some of the computational capabilities of differentiable
neural computers with a model that can be trained very efficiently, namely an
echo state network with an explicit memory without interference. This extension
enables echo state networks to recognize all regular languages, including those
that contractive echo state networks provably can not recognize. Further, we
demonstrate experimentally that our model performs comparably to its
fully-trained deep version on several typical benchmark tasks for
differentiable neural computers.
