Generating Multi-Agent Trajectories using Programmatic Weak Supervision
  We study the problem of training sequential generative models for capturing
coordinated multi-agent trajectory behavior, such as offensive basketball
gameplay. When modeling such settings, it is often beneficial to design
hierarchical models that can capture long-term coordination using intermediate
variables. Furthermore, these intermediate variables should capture interesting
high-level behavioral semantics in an interpretable and manipulatable way. We
present a hierarchical framework that can effectively learn such sequential
generative models. Our approach is inspired by recent work on leveraging
programmatically produced weak labels, which we extend to the spatiotemporal
regime. In addition to synthetic settings, we show how to instantiate our
framework to effectively model complex interactions between basketball players
and generate realistic multi-agent trajectories of basketball gameplay over
long time periods. We validate our approach using both quantitative and
qualitative evaluations, including a user study comparison conducted with
professional sports analysts.
