Image-to-Lidar Self-Supervised Distillation for Autonomous Driving Data
  Segmenting or detecting objects in sparse Lidar point clouds are two
important tasks in autonomous driving to allow a vehicle to act safely in its
3D environment. The best performing methods in 3D semantic segmentation or
object detection rely on a large amount of annotated data. Yet annotating 3D
Lidar data for these tasks is tedious and costly. In this context, we propose a
self-supervised pre-training method for 3D perception models that is tailored
to autonomous driving data. Specifically, we leverage the availability of
synchronized and calibrated image and Lidar sensors in autonomous driving
setups for distilling self-supervised pre-trained image representations into 3D
models. Hence, our method does not require any point cloud nor image
annotations. The key ingredient of our method is the use of superpixels which
are used to pool 3D point features and 2D pixel features in visually similar
regions. We then train a 3D network on the self-supervised task of matching
these pooled point features with the corresponding pooled image pixel features.
The advantages of contrasting regions obtained by superpixels are that: (1)
grouping together pixels and points of visually coherent regions leads to a
more meaningful contrastive task that produces features well adapted to 3D
semantic segmentation and 3D object detection; (2) all the different regions
have the same weight in the contrastive loss regardless of the number of 3D
points sampled in these regions; (3) it mitigates the noise produced by
incorrect matching of points and pixels due to occlusions between the different
sensors. Extensive experiments on autonomous driving datasets demonstrate the
ability of our image-to-Lidar distillation strategy to produce 3D
representations that transfer well on semantic segmentation and object
detection tasks.
