Unbiased Self-Play
  We present a general optimization framework for emergent belief-state
representation without any supervision. We employed the common configuration of
multiagent reinforcement learning and communication to improve exploration
coverage over an environment by leveraging the knowledge of each agent. In this
paper, we obtained that recurrent neural nets (RNNs) with shared weights are
highly biased in partially observable environments because of their
noncooperativity. To address this, we designated an unbiased version of
self-play via mechanism design, also known as reverse game theory, to clarify
unbiased knowledge at the Bayesian Nash equilibrium. The key idea is to add
imaginary rewards using the peer prediction mechanism, i.e., a mechanism for
mutually criticizing information in a decentralized environment. Numerical
analyses, including StarCraft exploration tasks with up to 20 agents and
off-the-shelf RNNs, demonstrate the state-of-the-art performance.
