Emergent Communication in a Multi-Modal, Multi-Step Referential Game
  Inspired by previous work on emergent communication in referential games, we
propose a novel multi-modal, multi-step referential game, where the sender and
receiver have access to distinct modalities of an object, and their information
exchange is bidirectional and of arbitrary duration. The multi-modal multi-step
setting allows agents to develop an internal communication significantly closer
to natural language, in that they share a single set of messages, and that the
length of the conversation may vary according to the difficulty of the task. We
examine these properties empirically using a dataset consisting of images and
textual descriptions of mammals, where the agents are tasked with identifying
the correct object. Our experiments indicate that a robust and efficient
communication protocol emerges, where gradual information exchange informs
better predictions and higher communication bandwidth improves generalization.
