Shared Cross-Modal Trajectory Prediction for Autonomous Driving
  Predicting future trajectories of traffic agents in highly interactive
environments is an essential and challenging problem for the safe operation of
autonomous driving systems. On the basis of the fact that self-driving vehicles
are equipped with various types of sensors (e.g., LiDAR scanner, RGB camera,
radar, etc.), we propose a Cross-Modal Embedding framework that aims to benefit
from the use of multiple input modalities. At training time, our model learns
to embed a set of complementary features in a shared latent space by jointly
optimizing the objective functions across different types of input data. At
test time, a single input modality (e.g., LiDAR data) is required to generate
predictions from the input perspective (i.e., in the LiDAR space), while taking
advantages from the model trained with multiple sensor modalities. An extensive
evaluation is conducted to show the efficacy of the proposed framework using
two benchmark driving datasets.
