Towards Cooperation in Sequential Prisoner's Dilemmas: a Deep Multiagent
  Reinforcement Learning Approach
  The Iterated Prisoner's Dilemma has guided research on social dilemmas for
decades. However, it distinguishes between only two atomic actions: cooperate
and defect. In real-world prisoner's dilemmas, these choices are temporally
extended and different strategies may correspond to sequences of actions,
reflecting grades of cooperation. We introduce a Sequential Prisoner's Dilemma
(SPD) game to better capture the aforementioned characteristics. In this work,
we propose a deep multiagent reinforcement learning approach that investigates
the evolution of mutual cooperation in SPD games. Our approach consists of two
phases. The first phase is offline: it synthesizes policies with different
cooperation degrees and then trains a cooperation degree detection network. The
second phase is online: an agent adaptively selects its policy based on the
detected degree of opponent cooperation. The effectiveness of our approach is
demonstrated in two representative SPD 2D games: the Apple-Pear game and the
Fruit Gathering game. Experimental results show that our strategy can avoid
being exploited by exploitative opponents and achieve cooperation with
cooperative opponents.
