Towards Understanding Adversarial Examples Systematically: Exploring
  Data Size, Task and Model Factors
  Most previous works usually explained adversarial examples from several
specific perspectives, lacking relatively integral comprehension about this
problem. In this paper, we present a systematic study on adversarial examples
from three aspects: the amount of training data, task-dependent and
model-specific factors. Particularly, we show that adversarial generalization
(i.e. test accuracy on adversarial examples) for standard training requires
more data than standard generalization (i.e. test accuracy on clean examples);
and uncover the global relationship between generalization and robustness with
respect to the data size especially when data is augmented by generative
models. This reveals the trade-off correlation between standard generalization
and robustness in limited training data regime and their consistency when data
size is large enough. Furthermore, we explore how different task-dependent and
model-specific factors influence the vulnerability of deep neural networks by
extensive empirical analysis. Relevant recommendations on defense against
adversarial attacks are provided as well. Our results outline a potential path
towards the luminous and systematic understanding of adversarial examples.
