Learning Diverse Policies in MOBA Games via Macro-Goals
  Recently, many researchers have made successful progress in building the AI
systems for MOBA-game-playing with deep reinforcement learning, such as on Dota
2 and Honor of Kings. Even though these AI systems have achieved or even
exceeded human-level performance, they still suffer from the lack of policy
diversity. In this paper, we propose a novel Macro-Goals Guided framework,
called MGG, to learn diverse policies in MOBA games. MGG abstracts strategies
as macro-goals from human demonstrations and trains a Meta-Controller to
predict these macro-goals. To enhance policy diversity, MGG samples macro-goals
from the Meta-Controller prediction and guides the training process towards
these goals. Experimental results on the typical MOBA game Honor of Kings
demonstrate that MGG can execute diverse policies in different matches and
lineups, and also outperform the state-of-the-art methods over 102 heroes.
