Software and application patterns for explanation methods
  Deep neural networks successfully pervaded many applications domains and are
increasingly used in critical decision processes. Understanding their workings
is desirable or even required to further foster their potential as well as to
access sensitive domains like medical applications or autonomous driving. One
key to this broader usage of explaining frameworks is the accessibility and
understanding of respective software. In this work we introduce software and
application patterns for explanation techniques that aim to explain individual
predictions of neural networks. We discuss how to code well-known algorithms
efficiently within deep learning software frameworks and describe how to embed
algorithms in downstream implementations. Building on this we show how
explanation methods can be used in applications to understand predictions for
miss-classified samples, to compare algorithms or networks, and to examine the
focus of networks. Furthermore, we review available open-source packages and
discuss challenges posed by complex and evolving neural network structures to
explanation algorithm development and implementations.
