Model-based Multi-Agent Reinforcement Learning with Cooperative
  Prioritized Sweeping
  We present a new model-based reinforcement learning algorithm, Cooperative
Prioritized Sweeping, for efficient learning in multi-agent Markov decision
processes. The algorithm allows for sample-efficient learning on large problems
by exploiting a factorization to approximate the value function. Our approach
only requires knowledge about the structure of the problem in the form of a
dynamic decision network. Using this information, our method learns a model of
the environment and performs temporal difference updates which affect multiple
joint states and actions at once. Batch updates are additionally performed
which efficiently back-propagate knowledge throughout the factored Q-function.
Our method outperforms the state-of-the-art algorithm sparse cooperative
Q-learning algorithm, both on the well-known SysAdmin benchmark and randomized
environments.
