VIB is Half Bayes
  In discriminative settings such as regression and classification there are
two random variables at play, the inputs X and the targets Y. Here, we
demonstrate that the Variational Information Bottleneck can be viewed as a
compromise between fully empirical and fully Bayesian objectives, attempting to
minimize the risks due to finite sampling of Y only. We argue that this
approach provides some of the benefits of Bayes while requiring only some of
the work.
