Deconfounded Representation Similarity for Comparison of Neural Networks
  Similarity metrics such as representational similarity analysis (RSA) and
centered kernel alignment (CKA) have been used to compare layer-wise
representations between neural networks. However, these metrics are confounded
by the population structure of data items in the input space, leading to
spuriously high similarity for even completely random neural networks and
inconsistent domain relations in transfer learning. We introduce a simple and
generally applicable fix to adjust for the confounder with covariate adjustment
regression, which retains the intuitive invariance properties of the original
similarity measures. We show that deconfounding the similarity metrics
increases the resolution of detecting semantically similar neural networks.
Moreover, in real-world applications, deconfounding improves the consistency of
representation similarities with domain similarities in transfer learning, and
increases correlation with out-of-distribution accuracy.
