Towards Open Ad Hoc Teamwork Using Graph-based Policy Learning
  Ad hoc teamwork is the challenging problem of designing an autonomous agent
which can adapt quickly to collaborate with teammates without prior
coordination mechanisms, including joint training. Prior work in this area has
focused on closed teams in which the number of agents is fixed. In this work,
we consider open teams by allowing agents with different fixed policies to
enter and leave the environment without prior notification. Our solution builds
on graph neural networks to learn agent models and joint-action value models
under varying team compositions. We contribute a novel action-value computation
that integrates the agent model and joint-action value model to produce
action-value estimates. We empirically demonstrate that our approach
successfully models the effects other agents have on the learner, leading to
policies that robustly adapt to dynamic team compositions and significantly
outperform several alternative methods.
