PTP: Parallelized Tracking and Prediction with Graph Neural Networks and
  Diversity Sampling
  Multi-object tracking (MOT) and trajectory prediction are two critical
components in modern 3D perception systems that require accurate modeling of
multi-agent interaction. We hypothesize that it is beneficial to unify both
tasks under one framework in order to learn a shared feature representation of
agent interaction. Furthermore, instead of performing tracking and prediction
sequentially which can propagate errors from tracking to prediction, we propose
a parallelized framework to mitigate the issue. Also, our parallel
track-forecast framework incorporates two additional novel computational units.
First, we use a feature interaction technique by introducing Graph Neural
Networks (GNNs) to capture the way in which agents interact with one another.
The GNN is able to improve discriminative feature learning for MOT association
and provide socially-aware contexts for trajectory prediction. Second, we use a
diversity sampling function to improve the quality and diversity of our
forecasted trajectories. The learned sampling function is trained to
efficiently extract a variety of outcomes from a generative trajectory
distribution and helps avoid the problem of generating duplicate trajectory
samples. We evaluate on KITTI and nuScenes datasets showing that our method
with socially-aware feature learning and diversity sampling achieves new
state-of-the-art performance on 3D MOT and trajectory prediction. Project
website is: https://www.xinshuoweng.com/projects/PTP
