Distributed Prioritized Experience Replay
  We propose a distributed architecture for deep reinforcement learning at
scale, that enables agents to learn effectively from orders of magnitude more
data than previously possible. The algorithm decouples acting from learning:
the actors interact with their own instances of the environment by selecting
actions according to a shared neural network, and accumulate the resulting
experience in a shared experience replay memory; the learner replays samples of
experience and updates the neural network. The architecture relies on
prioritized experience replay to focus only on the most significant data
generated by the actors. Our architecture substantially improves the state of
the art on the Arcade Learning Environment, achieving better final performance
in a fraction of the wall-clock training time.
