Experience Sharing Between Cooperative Reinforcement Learning Agents
  The idea of experience sharing between cooperative agents naturally emerges
from our understanding of how humans learn. Our evolution as a species is
tightly linked to the ability to exchange learned knowledge with one another.
It follows that experience sharing (ES) between autonomous and independent
agents could become the key to accelerate learning in cooperative multiagent
settings. We investigate if randomly selecting experiences to share can
increase the performance of deep reinforcement learning agents, and propose
three new methods for selecting experiences to accelerate the learning process.
Firstly, we introduce Focused ES, which prioritizes unexplored regions of the
state space. Secondly, we present Prioritized ES, in which temporal-difference
error is used as a measure of priority. Finally, we devise Focused Prioritized
ES, which combines both previous approaches. The methods are empirically
validated in a control problem. While sharing randomly selected experiences
between two Deep Q-Network agents shows no improvement over a single agent
baseline, we show that the proposed ES methods can successfully outperform the
baseline. In particular, the Focused ES accelerates learning by a factor of 2,
reducing by 51% the number of episodes required to complete the task.
