StarCraft Micromanagement with Reinforcement Learning and Curriculum
  Transfer Learning
  Real-time strategy games have been an important field of game artificial
intelligence in recent years. This paper presents a reinforcement learning and
curriculum transfer learning method to control multiple units in StarCraft
micromanagement. We define an efficient state representation, which breaks down
the complexity caused by the large state space in the game environment. Then a
parameter sharing multi-agent gradientdescent Sarsa({\lambda}) (PS-MAGDS)
algorithm is proposed to train the units. The learning policy is shared among
our units to encourage cooperative behaviors. We use a neural network as a
function approximator to estimate the action-value function, and propose a
reward function to help units balance their move and attack. In addition, a
transfer learning method is used to extend our model to more difficult
scenarios, which accelerates the training process and improves the learning
performance. In small scale scenarios, our units successfully learn to combat
and defeat the built-in AI with 100% win rates. In large scale scenarios,
curriculum transfer learning method is used to progressively train a group of
units, and shows superior performance over some baseline methods in target
scenarios. With reinforcement learning and curriculum transfer learning, our
units are able to learn appropriate strategies in StarCraft micromanagement
scenarios.
