TStarBot-X: An Open-Sourced and Comprehensive Study for Efficient League
  Training in StarCraft II Full Game
  StarCraft, one of the most difficult esport games with long-standing history
of professional tournaments, has attracted generations of players and fans, and
also, intense attentions in artificial intelligence research. Recently,
Google's DeepMind announced AlphaStar, a grandmaster level AI in StarCraft II
that can play with humans using comparable action space and operations. In this
paper, we introduce a new AI agent, named TStarBot-X, that is trained under
orders of less computations and can play competitively with expert human
players. TStarBot-X takes advantage of important techniques introduced in
AlphaStar, and also benefits from substantial innovations including new league
training methods, novel multi-agent roles, rule-guided policy search,
stabilized policy improvement, lightweight neural network architecture, and
importance sampling in imitation learning, etc. We show that with orders of
less computation scale, a faithful reimplementation of AlphaStar's methods can
not succeed and the proposed techniques are necessary to ensure TStarBot-X's
competitive performance. We reveal all technical details that are complementary
to those mentioned in AlphaStar, showing the most sensitive parts in league
training, reinforcement learning and imitation learning that affect the
performance of the agents. Most importantly, this is an open-sourced study that
all codes and resources (including the trained model parameters) are publicly
accessible via \url{https://github.com/tencent-ailab/tleague_projpage}. We
expect this study could be beneficial for both academic and industrial future
research in solving complex problems like StarCraft, and also, might provide a
sparring partner for all StarCraft II players and other AI agents.
