Mapping DNN Embedding Manifolds for Network Generalization Prediction
  Understanding Deep Neural Network (DNN) performance in changing conditions is
essential for deploying DNNs in safety critical applications with unconstrained
environments, e.g., perception for self-driving vehicles or medical image
analysis. Recently, the task of Network Generalization Prediction (NGP) has
been proposed to predict how a DNN will generalize in a new operating domain.
Previous NGP approaches have relied on labeled metadata and known distributions
for the new operating domains. In this study, we propose the first NGP approach
that predicts DNN performance based solely on how unlabeled images from an
external operating domain map in the DNN embedding space. We demonstrate this
technique for pedestrian, melanoma, and animal classification tasks and show
state of the art NGP in 13 of 15 NGP tasks without requiring domain knowledge.
Additionally, we show that our NGP embedding maps can be used to identify
misclassified images when the DNN performance is poor.
