Learning to Learn from Mistakes: Robust Optimization for Adversarial
  Noise
  Sensitivity to adversarial noise hinders deployment of machine learning
algorithms in security-critical applications. Although many adversarial
defenses have been proposed, robustness to adversarial noise remains an open
problem. The most compelling defense, adversarial training, requires a
substantial increase in processing time and it has been shown to overfit on the
training data. In this paper, we aim to overcome these limitations by training
robust models in low data regimes and transfer adversarial knowledge between
different models. We train a meta-optimizer which learns to robustly optimize a
model using adversarial examples and is able to transfer the knowledge learned
to new models, without the need to generate new adversarial examples.
Experimental results show the meta-optimizer is consistent across different
architectures and data sets, suggesting it is possible to automatically patch
adversarial vulnerabilities.
