General Pitfalls of Model-Agnostic Interpretation Methods for Machine
  Learning Models
  An increasing number of model-agnostic interpretation techniques for machine
learning (ML) models such as partial dependence plots (PDP), permutation
feature importance (PFI) and Shapley values provide insightful model
interpretations, but can lead to wrong conclusions if applied incorrectly. We
highlight many general pitfalls of ML model interpretation, such as using
interpretation techniques in the wrong context, interpreting models that do not
generalize well, ignoring feature dependencies, interactions, uncertainty
estimates and issues in high-dimensional settings, or making unjustified causal
interpretations, and illustrate them with examples. We focus on pitfalls for
global methods that describe the average model behavior, but many pitfalls also
apply to local methods that explain individual predictions. Our paper addresses
ML practitioners by raising awareness of pitfalls and identifying solutions for
correct model interpretation, but also addresses ML researchers by discussing
open issues for further research.
