Deep RTS: A Game Environment for Deep Reinforcement Learning in
  Real-Time Strategy Games
  Reinforcement learning (RL) is an area of research that has blossomed
tremendously in recent years and has shown remarkable potential for artificial
intelligence based opponents in computer games. This success is primarily due
to the vast capabilities of convolutional neural networks, that can extract
useful features from noisy and complex data. Games are excellent tools to test
and push the boundaries of novel RL algorithms because they give valuable
insight into how well an algorithm can perform in isolated environments without
the real-life consequences. Real-time strategy games (RTS) is a genre that has
tremendous complexity and challenges the player in short and long-term
planning. There is much research that focuses on applied RL in RTS games, and
novel advances are therefore anticipated in the not too distant future.
However, there are to date few environments for testing RTS AIs. Environments
in the literature are often either overly simplistic, such as microRTS, or
complex and without the possibility for accelerated learning on consumer
hardware like StarCraft II. This paper introduces the Deep RTS game environment
for testing cutting-edge artificial intelligence algorithms for RTS games. Deep
RTS is a high-performance RTS game made specifically for artificial
intelligence research. It supports accelerated learning, meaning that it can
learn at a magnitude of 50 000 times faster compared to existing RTS games.
Deep RTS has a flexible configuration, enabling research in several different
RTS scenarios, including partially observable state-spaces and map complexity.
We show that Deep RTS lives up to our promises by comparing its performance
with microRTS, ELF, and StarCraft II on high-end consumer hardware. Using Deep
RTS, we show that a Deep Q-Network agent beats random-play agents over 70% of
the time. Deep RTS is publicly available at https://github.com/cair/DeepRTS.
