Exploiting Semantic Epsilon Greedy Exploration Strategy in Multi-Agent
  Reinforcement Learning
  Multi-agent reinforcement learning (MARL) can model many real world
applications. However, many MARL approaches rely on epsilon greedy for
exploration, which may discourage visiting advantageous states in hard
scenarios. In this paper, we propose a new approach QMIX(SEG) for tackling
MARL. It makes use of the value function factorization method QMIX to train
per-agent policies and a novel Semantic Epsilon Greedy (SEG) exploration
strategy. SEG is a simple extension to the conventional epsilon greedy
exploration strategy, yet it is experimentally shown to greatly improve the
performance of MARL. We first cluster actions into groups of actions with
similar effects and then use the groups in a bi-level epsilon greedy
exploration hierarchy for action selection. We argue that SEG facilitates
semantic exploration by exploring in the space of groups of actions, which have
richer semantic meanings than atomic actions. Experiments show that QMIX(SEG)
largely outperforms QMIX and leads to strong performance competitive with
current state-of-the-art MARL approaches on the StarCraft Multi-Agent Challenge
(SMAC) benchmark.
