Instance-wise or Class-wise? A Tale of Neighbor Shapley for
  Concept-based Explanation
  Deep neural networks have demonstrated remarkable performance in many
data-driven and prediction-oriented applications, and sometimes even perform
better than humans. However, their most significant drawback is the lack of
interpretability, which makes them less attractive in many real-world
applications. When relating to the moral problem or the environmental factors
that are uncertain such as crime judgment, financial analysis, and medical
diagnosis, it is essential to mine the evidence for the model's prediction
(interpret model knowledge) to convince humans. Thus, investigating how to
interpret model knowledge is of paramount importance for both academic research
and real applications.
