Learning to Reason in Round-based Games: Multi-task Sequence Generation
  for Purchasing Decision Making in First-person Shooters
  Sequential reasoning is a complex human ability, with extensive previous
research focusing on gaming AI in a single continuous game, round-based
decision makings extending to a sequence of games remain less explored.
Counter-Strike: Global Offensive (CS:GO), as a round-based game with abundant
expert demonstrations, provides an excellent environment for multi-player
round-based sequential reasoning. In this work, we propose a Sequence Reasoner
with Round Attribute Encoder and Multi-Task Decoder to interpret the strategies
behind the round-based purchasing decisions. We adopt few-shot learning to
sample multiple rounds in a match, and modified model agnostic meta-learning
algorithm Reptile for the meta-learning loop. We formulate each round as a
multi-task sequence generation problem. Our state representations combine
action encoder, team encoder, player features, round attribute encoder, and
economy encoders to help our agent learn to reason under this specific
multi-player round-based scenario. A complete ablation study and comparison
with the greedy approach certify the effectiveness of our model. Our research
will open doors for interpretable AI for understanding episodic and long-term
purchasing strategies beyond the gaming community.
