Learning to Schedule Communication in Multi-agent Reinforcement Learning
  Many real-world reinforcement learning tasks require multiple agents to make
sequential decisions under the agents' interaction, where well-coordinated
actions among the agents are crucial to achieve the target goal better at these
tasks. One way to accelerate the coordination effect is to enable multiple
agents to communicate with each other in a distributed manner and behave as a
group. In this paper, we study a practical scenario when (i) the communication
bandwidth is limited and (ii) the agents share the communication medium so that
only a restricted number of agents are able to simultaneously use the medium,
as in the state-of-the-art wireless networking standards. This calls for a
certain form of communication scheduling. In that regard, we propose a
multi-agent deep reinforcement learning framework, called SchedNet, in which
agents learn how to schedule themselves, how to encode the messages, and how to
select actions based on received messages. SchedNet is capable of deciding
which agents should be entitled to broadcasting their (encoded) messages, by
learning the importance of each agent's partially observed information. We
evaluate SchedNet against multiple baselines under two different applications,
namely, cooperative communication and navigation, and predator-prey. Our
experiments show a non-negligible performance gap between SchedNet and other
mechanisms such as the ones without communication and with vanilla scheduling
methods, e.g., round robin, ranging from 32% to 43%.
