Signal Strength and Noise Drive Feature Preference in CNN Image
  Classifiers
  Feature preference in Convolutional Neural Network (CNN) image classifiers is
integral to their decision making process, and while the topic has been well
studied, it is still not understood at a fundamental level. We test a range of
task relevant feature attributes (including shape, texture, and color) with
varying degrees of signal and noise in highly controlled CNN image
classification experiments using synthetic datasets to determine feature
preferences. We find that CNNs will prefer features with stronger signal
strength and lower noise irrespective of whether the feature is texture, shape,
or color. This provides guidance for a predictive model for task relevant
feature preferences, demonstrates pathways for bias in machine models that can
be avoided with careful controls on experimental setup, and suggests that
comparisons between how humans and machines prefer task relevant features in
vision classification tasks should be revisited. Code to reproduce experiments
in this paper can be found at
\url{https://github.com/mwolff31/signal_preference}.
