DARL1N: Distributed multi-Agent Reinforcement Learning with One-hop
  Neighbors
  Most existing multi-agent reinforcement learning (MARL) methods are limited
in the scale of problems they can handle. Particularly, with the increase of
the number of agents, their training costs grow exponentially. In this paper,
we address this limitation by introducing a scalable MARL method called
Distributed multi-Agent Reinforcement Learning with One-hop Neighbors (DARL1N).
DARL1N is an off-policy actor-critic method that breaks the curse of
dimensionality by decoupling the global interactions among agents and
restricting information exchanges to one-hop neighbors. Each agent optimizes
its action value and policy functions over a one-hop neighborhood,
significantly reducing the learning complexity, yet maintaining expressiveness
by training with varying numbers and states of neighbors. This structure allows
us to formulate a distributed learning framework to further speed up the
training procedure. Comparisons with state-of-the-art MARL methods show that
DARL1N significantly reduces training time without sacrificing policy quality
and is scalable as the number of agents increases.
