Towards Safe, Real-Time Systems: Stereo vs Images and LiDAR for 3D
  Object Detection
  As object detectors rapidly improve, attention has expanded past image-only
networks to include a range of 3D and multimodal frameworks, especially ones
that incorporate LiDAR. However, due to cost, logistics, and even some safety
considerations, stereo can be an appealing alternative. Towards understanding
the efficacy of stereo as a replacement for monocular input or LiDAR in object
detectors, we show that multimodal learning with traditional disparity
algorithms can improve image-based results without increasing the number of
parameters, and that learning over stereo error can impart similar 3D
localization power to LiDAR in certain contexts. Furthermore, doing so also has
calibration benefits with respect to image-only methods. We benchmark on the
public dataset KITTI, and in doing so, reveal a few small but common
algorithmic mistakes currently used in computing metrics on that set, and offer
efficient, provably correct alternatives.
