Guidelines and evaluation for clinical explainable AI on medical image
  analysis
  Explainable artificial intelligence (XAI) is essential for enabling clinical
users to get informed decision support from AI and comply with evidence-based
medical practice. Applying XAI in clinical settings requires proper evaluation
criteria to ensure the explanation technique is both technically sound and
clinically useful, but specific support is lacking to achieve this goal. To
bridge the research gap, we propose the Clinical XAI Guidelines that consist of
five criteria a clinical XAI needs to be optimized for. The guidelines
recommend choosing an explanation form based on Guideline 1 (G1)
Understandability and G2 Clinical relevance. For the chosen explanation form,
its specific XAI technique should be optimized for G3 Truthfulness, G4
Informative plausibility, and G5 Computational efficiency.
  Following the guidelines, we conducted a systematic evaluation on a novel
problem of multi-modal medical image explanation with two clinical tasks, and
proposed new evaluation metrics accordingly. The evaluated 16 commonly-used
heatmap XAI techniques were not suitable for clinical use due to their failure
in \textbf{G3} and \textbf{G4}. Our evaluation demonstrated the use of Clinical
XAI Guidelines to support the design and evaluation for clinically viable XAI.
