Multi-View Fusion of Sensor Data for Improved Perception and Prediction
  in Autonomous Driving
  We present an end-to-end method for object detection and trajectory
prediction utilizing multi-view representations of LiDAR returns and camera
images. In this work, we recognize the strengths and weaknesses of different
view representations, and we propose an efficient and generic fusing method
that aggregates benefits from all views. Our model builds on a state-of-the-art
Bird's-Eye View (BEV) network that fuses voxelized features from a sequence of
historical LiDAR data as well as rasterized high-definition map to perform
detection and prediction tasks. We extend this model with additional LiDAR
Range-View (RV) features that use the raw LiDAR information in its native,
non-quantized representation. The RV feature map is projected into BEV and
fused with the BEV features computed from LiDAR and high-definition map. The
fused features are then further processed to output the final detections and
trajectories, within a single end-to-end trainable network. In addition, the RV
fusion of LiDAR and camera is performed in a straightforward and
computationally efficient manner using this framework. The proposed multi-view
fusion approach improves the state-of-the-art on proprietary large-scale
real-world data collected by a fleet of self-driving vehicles, as well as on
the public nuScenes data set with minimal increases on the computational cost.
