On the Effects of Data Distortion on Model Analysis and Training
  Data modification can introduce artificial information. It is often assumed
that the resulting artefacts are detrimental to training, whilst being
negligible when analysing models. We investigate these assumptions and conclude
that in some cases they are unfounded and lead to incorrect results.
Specifically, we show current shape bias identification methods and occlusion
robustness measures are biased and propose a fairer alternative for the latter.
Subsequently, through a series of experiments we seek to correct and strengthen
the community's perception of how distorting data affects learning. Based on
our empirical results we argue that the impact of the artefacts must be
understood and exploited rather than eliminated.
