Generalisation in humans and deep neural networks
  We compare the robustness of humans and current convolutional deep neural
networks (DNNs) on object recognition under twelve different types of image
degradations. First, using three well known DNNs (ResNet-152, VGG-19,
GoogLeNet) we find the human visual system to be more robust to nearly all of
the tested image manipulations, and we observe progressively diverging
classification error-patterns between humans and DNNs when the signal gets
weaker. Secondly, we show that DNNs trained directly on distorted images
consistently surpass human performance on the exact distortion types they were
trained on, yet they display extremely poor generalisation abilities when
tested on other distortion types. For example, training on salt-and-pepper
noise does not imply robustness on uniform white noise and vice versa. Thus,
changes in the noise distribution between training and testing constitutes a
crucial challenge to deep learning vision systems that can be systematically
addressed in a lifelong machine learning approach. Our new dataset consisting
of 83K carefully measured human psychophysical trials provide a useful
reference for lifelong robustness against image degradations set by the human
visual system.
