Parallel Knowledge Transfer in Multi-Agent Reinforcement Learning
  Multi-agent reinforcement learning is a standard framework for modeling
multi-agent interactions applied in real-world scenarios. Inspired by
experience sharing in human groups, learning knowledge parallel reusing between
agents can potentially promote team learning performance, especially in
multi-task environments. When all agents interact with the environment and
learn simultaneously, how each independent agent selectively learns from other
agents' behavior knowledge is a problem that we need to solve. This paper
proposes a novel knowledge transfer framework in MARL, PAT (Parallel
Attentional Transfer). We design two acting modes in PAT, student mode and
self-learning mode. Each agent in our approach trains a decentralized student
actor-critic to determine its acting mode at each time step. When agents are
unfamiliar with the environment, the shared attention mechanism in student mode
effectively selects learning knowledge from other agents to decide agents'
actions. PAT outperforms state-of-the-art empirical evaluation results against
the prior advising approaches. Our approach not only significantly improves
team learning rate and global performance, but also is flexible and
transferable to be applied in various multi-agent systems.
