NeuPL: Neural Population Learning
  Learning in strategy games (e.g. StarCraft, poker) requires the discovery of
diverse policies. This is often achieved by iteratively training new policies
against existing ones, growing a policy population that is robust to exploit.
This iterative approach suffers from two issues in real-world games: a) under
finite budget, approximate best-response operators at each iteration needs
truncating, resulting in under-trained good-responses populating the
population; b) repeated learning of basic skills at each iteration is wasteful
and becomes intractable in the presence of increasingly strong opponents. In
this work, we propose Neural Population Learning (NeuPL) as a solution to both
issues. NeuPL offers convergence guarantees to a population of best-responses
under mild assumptions. By representing a population of policies within a
single conditional model, NeuPL enables transfer learning across policies.
Empirically, we show the generality, improved performance and efficiency of
NeuPL across several test domains. Most interestingly, we show that novel
strategies become more accessible, not less, as the neural population expands.
