Adversarial Information Bottleneck
  The information bottleneck (IB) principle has been adopted to explain deep
learning in terms of information compression and prediction, which are balanced
by a trade-off hyperparameter. How to optimize the IB principle for better
robustness and figure out the effects of compression through the trade-off
hyperparameter are two challenging problems. Previous methods attempted to
optimize the IB principle by introducing random noise into learning the
representation and achieved state-of-the-art performance in the nuisance
information compression and semantic information extraction. However, their
performance on resisting adversarial perturbations is far less impressive. To
this end, we propose an adversarial information bottleneck (AIB) method without
any explicit assumptions about the underlying distribution of the
representations, which can be optimized effectively by solving a Min-Max
optimization problem. Numerical experiments on synthetic and real-world
datasets demonstrate its effectiveness on learning more invariant
representations and mitigating adversarial perturbations compared to several
competing IB methods. In addition, we analyse the adversarial robustness of
diverse IB methods contrasting with their IB curves, and reveal that IB models
with the hyperparameter $\beta$ corresponding to the knee point in the IB curve
achieve the best trade-off between compression and prediction, and has best
robustness against various attacks.
