Optimizing Information Loss Towards Robust Neural Networks
  Neural Networks (NNs) are vulnerable to adversarial examples. Such inputs
differ only slightly from their benign counterparts yet provoke
misclassifications of the attacked NNs. The required perturbations to craft the
examples are often negligible and even human imperceptible. To protect deep
learning-based systems from such attacks, several countermeasures have been
proposed with adversarial training still being considered the most effective.
Here, NNs are iteratively retrained using adversarial examples forming a
computational expensive and time consuming process often leading to a
performance decrease. To overcome the downsides of adversarial training while
still providing a high level of security, we present a new training approach we
call \textit{entropic retraining}. Based on an information-theoretic-inspired
analysis, entropic retraining mimics the effects of adversarial training
without the need of the laborious generation of adversarial examples. We
empirically show that entropic retraining leads to a significant increase in
NNs' security and robustness while only relying on the given original data.
With our prototype implementation we validate and show the effectiveness of our
approach for various NN architectures and data sets.
