Policy learning with partial observation and mechanical constraints for
  multi-person modeling
  Extracting the rules of real-world biological multi-agent behaviors is a
current challenge in various scientific and engineering fields. Biological
agents generally have limited observation and mechanical constraints; however,
most of the conventional data-driven models ignore such assumptions, resulting
in lack of biological plausibility and model interpretability for behavioral
analyses in biological and cognitive science. Here we propose sequential
generative models with partial observation and mechanical constraints, which
can visualize whose information the agents utilize and can generate
biologically plausible actions. We formulate this as a decentralized
multi-agent imitation learning problem, leveraging binary partial observation
models with a Gumbel-Softmax reparameterization and policy models based on
hierarchical variational recurrent neural networks with physical and
biomechanical constraints. We investigate the empirical performances using
real-world multi-person motion datasets from basketball and soccer games.
