Deep Probabilistic Ensembles: Approximate Variational Inference through
  KL Regularization
  In this paper, we introduce Deep Probabilistic Ensembles (DPEs), a scalable
technique that uses a regularized ensemble to approximate a deep Bayesian
Neural Network (BNN). We do so by incorporating a KL divergence penalty term
into the training objective of an ensemble, derived from the evidence lower
bound used in variational inference. We evaluate the uncertainty estimates
obtained from our models for active learning on visual classification. Our
approach steadily improves upon active learning baselines as the annotation
budget is increased.
