Anti-efficient encoding in emergent communication
  Despite renewed interest in emergent language simulations with neural
networks, little is known about the basic properties of the induced code, and
how they compare to human language. One fundamental characteristic of the
latter, known as Zipf's Law of Abbreviation (ZLA), is that more frequent words
are efficiently associated to shorter strings. We study whether the same
pattern emerges when two neural networks, a "speaker" and a "listener", are
trained to play a signaling game. Surprisingly, we find that networks develop
an \emph{anti-efficient} encoding scheme, in which the most frequent inputs are
associated to the longest messages, and messages in general are skewed towards
the maximum length threshold. This anti-efficient code appears easier to
discriminate for the listener, and, unlike in human communication, the speaker
does not impose a contrasting least-effort pressure towards brevity. Indeed,
when the cost function includes a penalty for longer messages, the resulting
message distribution starts respecting ZLA. Our analysis stresses the
importance of studying the basic features of emergent communication in a highly
controlled setup, to ensure the latter will not strand too far from human
language. Moreover, we present a concrete illustration of how different
functional pressures can lead to successful communication codes that lack basic
properties of human language, thus highlighting the role such pressures play in
the latter.
