Robust Learning via Ensemble Density Propagation in Deep Neural Networks
  Learning in uncertain, noisy, or adversarial environments is a challenging
task for deep neural networks (DNNs). We propose a new theoretically grounded
and efficient approach for robust learning that builds upon Bayesian estimation
and Variational Inference. We formulate the problem of density propagation
through layers of a DNN and solve it using an Ensemble Density Propagation
(EnDP) scheme. The EnDP approach allows us to propagate moments of the
variational probability distribution across the layers of a Bayesian DNN,
enabling the estimation of the mean and covariance of the predictive
distribution at the output of the model. Our experiments using MNIST and
CIFAR-10 datasets show a significant improvement in the robustness of the
trained models to random noise and adversarial attacks.
