Emergent Discrete Communication in Semantic Spaces
  Neural agents trained in reinforcement learning settings can learn to
communicate among themselves via discrete tokens, accomplishing as a team what
agents would be unable to do alone. However, the current standard of using
one-hot vectors as discrete communication tokens prevents agents from acquiring
more desirable aspects of communication such as zero-shot understanding.
Inspired by word embedding techniques from natural language processing, we
propose neural agent architectures that enables them to communicate via
discrete tokens derived from a learned, continuous space. We show in a decision
theoretic framework that our technique optimizes communication over a wide
range of scenarios, whereas one-hot tokens are only optimal under restrictive
assumptions. In self-play experiments, we validate that our trained agents
learn to cluster tokens in semantically-meaningful ways, allowing them
communicate in noisy environments where other techniques fail. Lastly, we
demonstrate both that agents using our method can effectively respond to novel
human communication and that humans can understand unlabeled emergent agent
communication, outperforming the use of one-hot communication.
