Learning to Infer Belief Embedded Communication
  In multi-agent collaboration problems with communication, an agent's ability
to encode their intention and interpret other agents' strategies is critical
for planning their future actions. This paper introduces a novel algorithm
called Intention Embedded Communication (IEC) to mimic an agent's language
learning ability. IEC contains a perception module for decoding other agents'
intentions in response to their past actions. It also includes a language
generation module for learning implicit grammar during communication with two
or more agents. Such grammar, by construction, should be compact for efficient
communication. Both modules undergo conjoint evolution - similar to an infant's
babbling that enables it to learn a language of choice by trial and error. We
utilised three multi-agent environments, namely predator/prey, traffic junction
and level-based foraging and illustrate that such a co-evolution enables us to
learn much quicker (50%) than state-of-the-art algorithms like MADDPG. Ablation
studies further show that disabling the inferring belief module, communication
module, and the hidden states reduces the model performance by 38%, 60% and
30%, respectively. Hence, we suggest that modelling other agents' behaviour
accelerates another agent to learn grammar and develop a language to
communicate efficiently. We evaluate our method on a set of cooperative
scenarios and show its superior performance to other multi-agent baselines. We
also demonstrate that it is essential for agents to reason about others' states
and learn this ability by continuous communication.
