Exploring the Interchangeability of CNN Embedding Spaces
  CNN feature spaces can be linearly mapped and consequently are often
interchangeable. This equivalence holds across variations in architectures,
training datasets, and network tasks. Specifically, we mapped between 10
image-classification CNNs and between 4 facial-recognition CNNs. When image
embeddings generated by one CNN are transformed into embeddings corresponding
to the feature space of a second CNN trained on the same task, their respective
image classification or face verification performance is largely preserved. For
CNNs trained to the same classes and sharing a common backend-logit (soft-max)
architecture, a linear-mapping may always be calculated directly from the
backend layer weights. However, the case of a closed-set analysis with perfect
knowledge of classifiers is limiting. Therefore, empirical methods of
estimating mappings are presented for both the closed-set image classification
task and the open-set task of face recognition. The results presented expose
the essentially interchangeable nature of CNNs embeddings for two important and
common recognition tasks. The implications are far-reaching, suggesting an
underlying commonality between representations learned by networks designed and
trained for a common task. One practical implication is that face embeddings
from some commonly used CNNs can be compared using these mappings.
