Skill Discovery of Coordination in Multi-agent Reinforcement Learning
  Unsupervised skill discovery drives intelligent agents to explore the unknown
environment without task-specific reward signal, and the agents acquire various
skills which may be useful when the agents adapt to new tasks. In this paper,
we propose "Multi-agent Skill Discovery"(MASD), a method for discovering skills
for coordination patterns of multiple agents. The proposed method aims to
maximize the mutual information between a latent code Z representing skills and
the combination of the states of all agents. Meanwhile it suppresses the
empowerment of Z on the state of any single agent by adversarial training. In
another word, it sets an information bottleneck to avoid empowerment
degeneracy. First we show the emergence of various skills on the level of
coordination in a general particle multi-agent environment. Second, we reveal
that the "bottleneck" prevents skills from collapsing to a single agent and
enhances the diversity of learned skills. Finally, we show the pretrained
policies have better performance on supervised RL tasks.
