Input Bias in Rectified Gradients and Modified Saliency Maps
  Interpretation and improvement of deep neural networks relies on better
understanding of their underlying mechanisms. In particular, gradients of
classes or concepts with respect to the input features (e.g., pixels in images)
are often used as importance scores or estimators, which are visualized in
saliency maps. Thus, a family of saliency methods provide an intuitive way to
identify input features with substantial influences on classifications or
latent concepts. Several modifications to conventional saliency maps, such as
Rectified Gradients and Layer-wise Relevance Propagation (LRP), have been
introduced to allegedly denoise and improve interpretability. While visually
coherent in certain cases, Rectified Gradients and other modified saliency maps
introduce a strong input bias (e.g., brightness in the RGB space) because of
inappropriate uses of the input features. We demonstrate that dark areas of an
input image are not highlighted by a saliency map using Rectified Gradients,
even if it is relevant for the class or concept. Even in the scaled images, the
input bias exists around an artificial point in color spectrum. Our
modification, which simply eliminates multiplication with input features,
removes this bias. This showcases how a visual criteria may not align with true
explainability of deep learning models.
