End-To-End Optimization of LiDAR Beam Configuration for 3D Object
  Detection and Localization
  Existing learning methods for LiDAR-based applications use 3D points scanned
under a pre-determined beam configuration, e.g., the elevation angles of beams
are often evenly distributed. Those fixed configurations are task-agnostic, so
simply using them can lead to sub-optimal performance. In this work, we take a
new route to learn to optimize the LiDAR beam configuration for a given
application. Specifically, we propose a reinforcement learning-based
learning-to-optimize (RL-L2O) framework to automatically optimize the beam
configuration in an end-to-end manner for different LiDAR-based applications.
The optimization is guided by the final performance of the target task and thus
our method can be integrated easily with any LiDAR-based application as a
simple drop-in module. The method is especially useful when a low-resolution
(low-cost) LiDAR is needed, for instance, for system deployment at a massive
scale. We use our method to search for the beam configuration of a
low-resolution LiDAR for two important tasks: 3D object detection and
localization. Experiments show that the proposed RL-L2O method improves the
performance in both tasks significantly compared to the baseline methods. We
believe that a combination of our method with the recent advances of
programmable LiDARs can start a new research direction for LiDAR-based active
perception. The code is publicly available at
https://github.com/vniclas/lidar_beam_selection
