Multi-Issue Bargaining With Deep Reinforcement Learning
  Negotiation is a process where agents aim to work through disputes and
maximize their surplus. As the use of deep reinforcement learning in bargaining
games is unexplored, this paper evaluates its ability to exploit, adapt, and
cooperate to produce fair outcomes. Two actor-critic networks were trained for
the bidding and acceptance strategy, against time-based agents, behavior-based
agents, and through self-play. Gameplay against these agents reveals three key
findings. 1) Neural agents learn to exploit time-based agents, achieving clear
transitions in decision preference values. The Cauchy distribution emerges as
suitable for sampling offers, due to its peaky center and heavy tails. The
kurtosis and variance sensitivity of the probability distributions used for
continuous control produce trade-offs in exploration and exploitation. 2)
Neural agents demonstrate adaptive behavior against different combinations of
concession, discount factors, and behavior-based strategies. 3) Most
importantly, neural agents learn to cooperate with other behavior-based agents,
in certain cases utilizing non-credible threats to force fairer results. This
bears similarities with reputation-based strategies in the evolutionary
dynamics, and departs from equilibria in classical game theory.
