Novel Change of Measure Inequalities with Applications to PAC-Bayesian
  Bounds and Monte Carlo Estimation
  We introduce several novel change of measure inequalities for two families of
divergences: $f$-divergences and $\alpha$-divergences. We show how the
variational representation for $f$-divergences leads to novel change of measure
inequalities. We also present a multiplicative change of measure inequality for
$\alpha$-divergences and a generalized version of Hammersley-Chapman-Robbins
inequality. Finally, we present several applications of our change of measure
inequalities, including PAC-Bayesian bounds for various classes of losses and
non-asymptotic intervals for Monte Carlo estimates.
