Creating Pro-Level AI for a Real-Time Fighting Game Using Deep
  Reinforcement Learning
  Reinforcement learning combined with deep neural networks has performed
remarkably well in many genres of games recently. It has surpassed human-level
performance in fixed game environments and turn-based two player board games.
However, to the best of our knowledge, current research has yet to produce a
result that has surpassed human-level performance in modern complex fighting
games. This is due to the inherent difficulties with real-time fighting games,
including: vast action spaces, action dependencies, and imperfect information.
We overcame these challenges and made 1v1 battle AI agents for the commercial
game "Blade & Soul". The trained agents competed against five professional
gamers and achieved a win rate of 62%. This paper presents a practical
reinforcement learning method that includes a novel self-play curriculum and
data skipping techniques. Through the curriculum, three different styles of
agents were created by reward shaping and were trained against each other.
Additionally, this paper suggests data skipping techniques that could increase
data efficiency and facilitate explorations in vast spaces. Since our method
can be generally applied to all two-player competitive games with vast action
spaces, we anticipate its application to game development including level
design and automated balancing.
