Learning how to explain neural networks: PatternNet and
  PatternAttribution
  DeConvNet, Guided BackProp, LRP, were invented to better understand deep
neural networks. We show that these methods do not produce the theoretically
correct explanation for a linear model. Yet they are used on multi-layer
networks with millions of parameters. This is a cause for concern since linear
models are simple neural networks. We argue that explanation methods for neural
nets should work reliably in the limit of simplicity, the linear models. Based
on our analysis of linear models we propose a generalization that yields two
explanation techniques (PatternNet and PatternAttribution) that are
theoretically sound for linear models and produce improved explanations for
deep networks.
