End-to-End Multi-Task Deep Learning and Model Based Control Algorithm
  for Autonomous Driving
  End-to-end driving with a deep learning neural network (DNN) has become a
rapidly growing paradigm of autonomous driving in industry and academia. Yet
safety measures and interpretability still pose challenges to this paradigm. We
propose an end-to-end driving algorithm that integrates multi-task DNN, path
prediction, and control models in a pipeline of data flow from sensory devices
through these models to driving decisions. It provides quantitative measures to
evaluate the holistic, dynamic, and real-time performance of end-to-end driving
systems, and thus allows to quantify their safety and interpretability. The DNN
is a modified UNet, a well known encoder-decoder neural network of semantic
segmentation. It consists of one segmentation, one regression, and two
classification tasks for lane segmentation, path prediction, and vehicle
controls. We present three variants of the modified UNet architecture having
different complexities, compare them on different tasks in four static measures
for both single and multi-task (MT) architectures, and then identify the best
one by two additional dynamic measures in real-time simulation. We also propose
a learning- and model-based longitudinal controller using model predictive
control method. With the Stanley lateral controller, our results show that
MTUNet outperforms an earlier modified UNet in terms of curvature and lateral
offset estimation on curvy roads at normal speed, which has been tested in a
real car driving on real roads.
