Learnable Bernoulli Dropout for Bayesian Deep Learning
  In this work, we propose learnable Bernoulli dropout (LBD), a new
model-agnostic dropout scheme that considers the dropout rates as parameters
jointly optimized with other model parameters. By probabilistic modeling of
Bernoulli dropout, our method enables more robust prediction and uncertainty
quantification in deep models. Especially, when combined with variational
auto-encoders (VAEs), LBD enables flexible semi-implicit posterior
representations, leading to new semi-implicit VAE~(SIVAE) models. We solve the
optimization for training with respect to the dropout parameters using
Augment-REINFORCE-Merge (ARM), an unbiased and low-variance gradient estimator.
Our experiments on a range of tasks show the superior performance of our
approach compared with other commonly used dropout schemes. Overall, LBD leads
to improved accuracy and uncertainty estimates in image classification and
semantic segmentation. Moreover, using SIVAE, we can achieve state-of-the-art
performance on collaborative filtering for implicit feedback on several public
datasets.
