TLeague: A Framework for Competitive Self-Play based Distributed
  Multi-Agent Reinforcement Learning
  Competitive Self-Play (CSP) based Multi-Agent Reinforcement Learning (MARL)
has shown phenomenal breakthroughs recently. Strong AIs are achieved for
several benchmarks, including Dota 2, Glory of Kings, Quake III, StarCraft II,
to name a few. Despite the success, the MARL training is extremely data
thirsty, requiring typically billions of (if not trillions of) frames be seen
from the environment during training in order for learning a high performance
agent. This poses non-trivial difficulties for researchers or engineers and
prevents the application of MARL to a broader range of real-world problems. To
address this issue, in this manuscript we describe a framework, referred to as
TLeague, that aims at large-scale training and implements several main-stream
CSP-MARL algorithms. The training can be deployed in either a single machine or
a cluster of hybrid machines (CPUs and GPUs), where the standard Kubernetes is
supported in a cloud native manner. TLeague achieves a high throughput and a
reasonable scale-up when performing distributed training. Thanks to the modular
design, it is also easy to extend for solving other multi-agent problems or
implementing and verifying MARL algorithms. We present experiments over
StarCraft II, ViZDoom and Pommerman to show the efficiency and effectiveness of
TLeague. The code is open-sourced and available at
https://github.com/tencent-ailab/tleague_projpage
