Adversarial Mixture Density Networks: Learning to Drive Safely from
  Collision Data
  Imitation learning has been widely used to learn control policies for
autonomous driving based on pre-recorded data. However, imitation learning
based policies have been shown to be susceptible to compounding errors when
encountering states outside of the training distribution. Further, these agents
have been demonstrated to be easily exploitable by adversarial road users
aiming to create collisions. To overcome these shortcomings, we introduce
Adversarial Mixture Density Networks (AMDN), which learns two distributions
from separate datasets. The first is a distribution of safe actions learned
from a dataset of naturalistic human driving. The second is a distribution
representing unsafe actions likely to lead to collision, learned from a dataset
of collisions. During training, we leverage these two distributions to provide
an additional loss based on the similarity of the two distributions. By
penalising the safe action distribution based on its similarity to the unsafe
action distribution when training on the collision dataset, a more robust and
safe control policy is obtained. We demonstrate the proposed AMDN approach in a
vehicle following use-case, and evaluate under naturalistic and adversarial
testing environments. We show that despite its simplicity, AMDN provides
significant benefits for the safety of the learned control policy, when
compared to pure imitation learning or standard mixture density network
approaches.
