I'm Sorry for Your Loss: Spectrally-Based Audio Distances Are Bad at
  Pitch
  Growing research demonstrates that synthetic failure modes imply poor
generalization. We compare commonly used audio-to-audio losses on a synthetic
benchmark, measuring the pitch distance between two stationary sinusoids. The
results are surprising: many have poor sense of pitch direction. These
shortcomings are exposed using simple rank assumptions. Our task is trivial for
humans but difficult for these audio distances, suggesting significant progress
can be made in self-supervised audio learning by improving current losses.
