Interpreting and Explaining Deep Neural Networks for Classification of
  Audio Signals
  Interpretability of deep neural networks is a recently emerging area of
machine learning research targeting a better understanding of how models
perform feature selection and derive their classification decisions. This paper
explores the interpretability of neural networks in the audio domain by using
the previously proposed technique of layer-wise relevance propagation (LRP). We
present a novel audio dataset of English spoken digits which we use for
classification tasks on spoken digits and speaker's gender. We use LRP to
identify relevant features for two neural network architectures that process
either waveform or spectrogram representations of the data. Based on the
relevance scores obtained from LRP, hypotheses about the neural networks'
feature selection are derived and subsequently tested through systematic
manipulations of the input data. The results confirm that the networks are
highly reliant on features marked as relevant by LRP.
