Know Where To Drop Your Weights: Towards Faster Uncertainty Estimation
  Estimating epistemic uncertainty of models used in low-latency applications
and Out-Of-Distribution samples detection is a challenge due to the
computationally demanding nature of uncertainty estimation techniques.
Estimating model uncertainty using approximation techniques like Monte Carlo
Dropout (MCD), DropConnect (MCDC) requires a large number of forward passes
through the network, rendering them inapt for low-latency applications. We
propose Select-DC which uses a subset of layers in a neural network to model
epistemic uncertainty with MCDC. Through our experiments, we show a significant
reduction in the GFLOPS required to model uncertainty, compared to Monte Carlo
DropConnect, with marginal trade-off in performance. We perform a suite of
experiments on CIFAR 10, CIFAR 100, and SVHN datasets with ResNet and VGG
models. We further show how applying DropConnect to various layers in the
network with different drop probabilities affects the networks performance and
the entropy of the predictive distribution.
