Grounding Representation Similarity with Statistical Testing
  To understand neural network behavior, recent works quantitatively compare
different networks' learned representations using canonical correlation
analysis (CCA), centered kernel alignment (CKA), and other dissimilarity
measures. Unfortunately, these widely used measures often disagree on
fundamental observations, such as whether deep networks differing only in
random initialization learn similar representations. These disagreements raise
the question: which, if any, of these dissimilarity measures should we believe?
We provide a framework to ground this question through a concrete test:
measures should have sensitivity to changes that affect functional behavior,
and specificity against changes that do not. We quantify this through a variety
of functional behaviors including probing accuracy and robustness to
distribution shift, and examine changes such as varying random initialization
and deleting principal components. We find that current metrics exhibit
different weaknesses, note that a classical baseline performs surprisingly
well, and highlight settings where all metrics appear to fail, thus providing a
challenge set for further improvement.
