Evaluating Uncertainty Estimation Methods on 3D Semantic Segmentation of
  Point Clouds
  Deep learning models are extensively used in various safety critical
applications. Hence these models along with being accurate need to be highly
reliable. One way of achieving this is by quantifying uncertainty. Bayesian
methods for UQ have been extensively studied for Deep Learning models applied
on images but have been less explored for 3D modalities such as point clouds
often used for Robots and Autonomous Systems. In this work, we evaluate three
uncertainty quantification methods namely Deep Ensembles, MC-Dropout and
MC-DropConnect on the DarkNet21Seg 3D semantic segmentation model and
comprehensively analyze the impact of various parameters such as number of
models in ensembles or forward passes, and drop probability values, on task
performance and uncertainty estimate quality. We find that Deep Ensembles
outperforms other methods in both performance and uncertainty metrics. Deep
ensembles outperform other methods by a margin of 2.4% in terms of mIOU, 1.3%
in terms of accuracy, while providing reliable uncertainty for decision making.
