Many Agent Reinforcement Learning Under Partial Observability
  Recent renewed interest in multi-agent reinforcement learning (MARL) has
generated an impressive array of techniques that leverage deep reinforcement
learning, primarily actor-critic architectures, and can be applied to a limited
range of settings in terms of observability and communication. However, a
continuing limitation of much of this work is the curse of dimensionality when
it comes to representations based on joint actions, which grow exponentially
with the number of agents. In this paper, we squarely focus on this challenge
of scalability. We apply the key insight of action anonymity, which leads to
permutation invariance of joint actions, to two recently presented deep MARL
algorithms, MADDPG and IA2C, and compare these instantiations to another recent
technique that leverages action anonymity, viz., mean-field MARL. We show that
our instantiations can learn the optimal behavior in a broader class of agent
networks than the mean-field method, using a recently introduced pragmatic
domain.
