Understanding Convolutional Networks with APPLE : Automatic Patch
  Pattern Labeling for Explanation
  With the success of deep learning, recent efforts have been focused on
analyzing how learned networks make their classifications. We are interested in
analyzing the network output based on the network structure and information
flow through the network layers. We contribute an algorithm for 1) analyzing a
deep network to find neurons that are 'important' in terms of the network
classification outcome, and 2)automatically labeling the patches of the input
image that activate these important neurons. We propose several measures of
importance for neurons and demonstrate that our technique can be used to gain
insight into, and explain how a network decomposes an image to make its final
classification.
