Explaining Neural Network Predictions for Functional Data Using
  Principal Component Analysis and Feature Importance
  Optical spectral-temporal signatures extracted from videos of explosions
provide information for identifying characteristics of the corresponding
explosive devices. Currently, the identification is done using heuristic
algorithms and direct subject matter expert review. An improvement in
predictive performance may be obtained by using machine learning, but this
application lends itself to high consequence national security decisions, so it
is not only important to provide high accuracy but clear explanations for the
predictions to garner confidence in the model. While much work has been done to
develop explainability methods for machine learning models, not much of the
work focuses on situations with input variables of the form of functional data
such optical spectral-temporal signatures. We propose a procedure for
explaining machine learning models fit using functional data that accounts for
the functional nature the data. Our approach makes use of functional principal
component analysis (fPCA) and permutation feature importance (PFI). fPCA is
used to transform the functions to create uncorrelated functional principal
components (fPCs). The model is trained using the fPCs as inputs, and PFI is
applied to identify the fPCs important to the model for prediction.
Visualizations are used to interpret the variability explained by the fPCs that
are found to be important by PFI to determine the aspects of the functions that
are important for prediction. We demonstrate the technique by explaining neural
networks fit to explosion optical spectral-temporal signatures for predicting
characteristics of the explosive devices.
