PettingZoo: Gym for Multi-Agent Reinforcement Learning
  This paper introduces the PettingZoo library and the accompanying Agent
Environment Cycle ("AEC") games model. PettingZoo is a library of diverse sets
of multi-agent environments with a universal, elegant Python API. PettingZoo
was developed with the goal of accelerating research in Multi-Agent
Reinforcement Learning ("MARL"), by making work more interchangeable,
accessible and reproducible akin to what OpenAI's Gym library did for
single-agent reinforcement learning. PettingZoo's API, while inheriting many
features of Gym, is unique amongst MARL APIs in that it's based around the
novel AEC games model. We argue, in part through case studies on major problems
in popular MARL environments, that the popular game models are poor conceptual
models of games commonly used in MARL and accordingly can promote confusing
bugs that are hard to detect, and that the AEC games model addresses these
problems.
