Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A
  Review
  Autonomous vehicles were experiencing rapid development in the past few
years. However, achieving full autonomy is not a trivial task, due to the
nature of the complex and dynamic driving environment. Therefore, autonomous
vehicles are equipped with a suite of different sensors to ensure robust,
accurate environmental perception. In particular, the camera-LiDAR fusion is
becoming an emerging research theme. However, so far there has been no critical
review that focuses on deep-learning-based camera-LiDAR fusion methods. To
bridge this gap and motivate future research, this paper devotes to review
recent deep-learning-based data fusion approaches that leverage both image and
point cloud. This review gives a brief overview of deep learning on image and
point cloud data processing. Followed by in-depth reviews of camera-LiDAR
fusion methods in depth completion, object detection, semantic segmentation,
tracking and online cross-sensor calibration, which are organized based on
their respective fusion levels. Furthermore, we compare these methods on
publicly available datasets. Finally, we identified gaps and over-looked
challenges between current academic researches and real-world applications.
Based on these observations, we provide our insights and point out promising
research directions.
