Emergence of Language with Multi-agent Games: Learning to Communicate
  with Sequences of Symbols
  Learning to communicate through interaction, rather than relying on explicit
supervision, is often considered a prerequisite for developing a general AI. We
study a setting where two agents engage in playing a referential game and, from
scratch, develop a communication protocol necessary to succeed in this game.
Unlike previous work, we require that messages they exchange, both at train and
test time, are in the form of a language (i.e. sequences of discrete symbols).
We compare a reinforcement learning approach and one using a differentiable
relaxation (straight-through Gumbel-softmax estimator) and observe that the
latter is much faster to converge and it results in more effective protocols.
Interestingly, we also observe that the protocol we induce by optimizing the
communication success exhibits a degree of compositionality and variability
(i.e. the same information can be phrased in different ways), both properties
characteristic of natural languages. As the ultimate goal is to ensure that
communication is accomplished in natural language, we also perform experiments
where we inject prior information about natural language into our model and
study properties of the resulting protocol.
