Using learned optimizers to make models robust to input noise
  State-of-the art vision models can achieve superhuman performance on image
classification tasks when testing and training data come from the same
distribution. However, when models are tested on corrupted images (e.g. due to
scale changes, translations, or shifts in brightness or contrast), performance
degrades significantly. Here, we explore the possibility of meta-training a
learned optimizer that can train image classification models such that they are
robust to common image corruptions. Specifically, we are interested training
models that are more robust to noise distributions not present in the training
data. We find that a learned optimizer meta-trained to produce models which are
robust to Gaussian noise trains models that are more robust to Gaussian noise
at other scales compared to traditional optimizers like Adam. The effect of
meta-training is more complicated when targeting a more general set of noise
distributions, but led to improved performance on half of held-out corruption
tasks. Our results suggest that meta-learning provides a novel approach for
studying and improving the robustness of deep learning models.
