A Comprehensive guide to Bayesian Convolutional Neural Network with
  Variational Inference
  Artificial Neural Networks are connectionist systems that perform a given
task by learning on examples without having prior knowledge about the task.
This is done by finding an optimal point estimate for the weights in every
node. Generally, the network using point estimates as weights perform well with
large datasets, but they fail to express uncertainty in regions with little or
no data, leading to overconfident decisions.
  In this paper, Bayesian Convolutional Neural Network (BayesCNN) using
Variational Inference is proposed, that introduces probability distribution
over the weights. Furthermore, the proposed BayesCNN architecture is applied to
tasks like Image Classification, Image Super-Resolution and Generative
Adversarial Networks. The results are compared to point-estimates based
architectures on MNIST, CIFAR-10 and CIFAR-100 datasets for Image
CLassification task, on BSD300 dataset for Image Super Resolution task and on
CIFAR10 dataset again for Generative Adversarial Network task.
  BayesCNN is based on Bayes by Backprop which derives a variational
approximation to the true posterior. We, therefore, introduce the idea of
applying two convolutional operations, one for the mean and one for the
variance. Our proposed method not only achieves performances equivalent to
frequentist inference in identical architectures but also incorporate a
measurement for uncertainties and regularisation. It further eliminates the use
of dropout in the model. Moreover, we predict how certain the model prediction
is based on the epistemic and aleatoric uncertainties and empirically show how
the uncertainty can decrease, allowing the decisions made by the network to
become more deterministic as the training accuracy increases. Finally, we
propose ways to prune the Bayesian architecture and to make it more
computational and time effective.
