Reliably Re-Acting to Partner's Actions with the Social Intrinsic
  Motivation of Transfer Empowerment
  We consider multi-agent reinforcement learning (MARL) for cooperative
communication and coordination tasks. MARL agents can be brittle because they
can overfit their training partners' policies. This overfitting can produce
agents that adopt policies that act under the expectation that other agents
will act in a certain way rather than react to their actions. Our objective is
to bias the learning process towards finding reactive strategies towards other
agents' behaviors. Our method, transfer empowerment, measures the potential
influence between agents' actions. Results from three simulated cooperation
scenarios support our hypothesis that transfer empowerment improves MARL
performance. We discuss how transfer empowerment could be a useful principle to
guide multi-agent coordination by ensuring reactiveness to one's partner.
