RGB and LiDAR fusion based 3D Semantic Segmentation for Autonomous
  Driving
  LiDAR has become a standard sensor for autonomous driving applications as
they provide highly precise 3D point clouds. LiDAR is also robust for low-light
scenarios at night-time or due to shadows where the performance of cameras is
degraded. LiDAR perception is gradually becoming mature for algorithms
including object detection and SLAM. However, semantic segmentation algorithm
remains to be relatively less explored. Motivated by the fact that semantic
segmentation is a mature algorithm on image data, we explore sensor fusion
based 3D segmentation. Our main contribution is to convert the RGB image to a
polar-grid mapping representation used for LiDAR and design early and mid-level
fusion architectures. Additionally, we design a hybrid fusion architecture that
combines both fusion algorithms. We evaluate our algorithm on KITTI dataset
which provides segmentation annotation for cars, pedestrians and cyclists. We
evaluate two state-of-the-art architectures namely SqueezeSeg and PointSeg and
improve the mIoU score by 10 % in both cases relative to the LiDAR only
baseline.
