Validation and generalization of pixel-wise relevance in convolutional
  neural networks trained for face classification
  The increased use of convolutional neural networks for face recognition in
science, governance, and broader society has created an acute need for methods
that can show how these 'black box' decisions are made. To be interpretable and
useful to humans, such a method should convey a model's learned classification
strategy in a way that is robust to random initializations or spurious
correlations in input data. To this end, we applied the decompositional
pixel-wise attribution method of layer-wise relevance propagation (LRP) to
resolve the decisions of several classes of VGG-16 models trained for face
recognition. We then quantified how these relevance measures vary with and
generalize across key model parameters, such as the pretraining dataset
(ImageNet or VGGFace), the finetuning task (gender or identity classification),
and random initializations of model weights. Using relevance-based image
masking, we find that relevance maps for face classification prove generally
stable across random initializations, and can generalize across finetuning
tasks. However, there is markedly less generalization across pretraining
datasets, indicating that ImageNet- and VGGFace-trained models sample face
information differently even as they achieve comparably high classification
performance. Fine-grained analyses of relevance maps across models revealed
asymmetries in generalization that point to specific benefits of choice
parameters, and suggest that it may be possible to find an underlying set of
important face image pixels that drive decisions across convolutional neural
networks and tasks. Finally, we evaluated model decision weighting against
human measures of similarity, providing a novel framework for interpreting face
recognition decisions across human and machine.
