Explaining Convolutional Neural Networks using Softmax Gradient
  Layer-wise Relevance Propagation
  Convolutional Neural Networks (CNN) have become state-of-the-art in the field
of image classification. However, not everything is understood about their
inner representations. This paper tackles the interpretability and
explainability of the predictions of CNNs for multi-class classification
problems. Specifically, we propose a novel visualization method of pixel-wise
input attribution called Softmax-Gradient Layer-wise Relevance Propagation
(SGLRP). The proposed model is a class discriminate extension to Deep Taylor
Decomposition (DTD) using the gradient of softmax to back propagate the
relevance of the output probability to the input image. Through qualitative and
quantitative analysis, we demonstrate that SGLRP can successfully localize and
attribute the regions on input images which contribute to a target object's
classification. We show that the proposed method excels at discriminating the
target objects class from the other possible objects in the images. We confirm
that SGLRP performs better than existing Layer-wise Relevance Propagation (LRP)
based methods and can help in the understanding of the decision process of
CNNs.
