Efficient Distributed Framework for Collaborative Multi-Agent
  Reinforcement Learning
  Multi-agent reinforcement learning for incomplete information environments
has attracted extensive attention from researchers. However, due to the slow
sample collection and poor sample exploration, there are still some problems in
multi-agent reinforcement learning, such as unstable model iteration and low
training efficiency. Moreover, most of the existing distributed framework are
proposed for single-agent reinforcement learning and not suitable for
multi-agent. In this paper, we design an distributed MARL framework based on
the actor-work-learner architecture. In this framework, multiple asynchronous
environment interaction modules can be deployed simultaneously, which greatly
improves the sample collection speed and sample diversity. Meanwhile, to make
full use of computing resources, we decouple the model iteration from
environment interaction, and thus accelerate the policy iteration. Finally, we
verified the effectiveness of propose framework in MaCA military simulation
environment and the SMAC 3D realtime strategy gaming environment with
imcomplete information characteristics.
