Harmonicity Plays a Critical Role in DNN Based Versus in
  Biologically-Inspired Monaural Speech Segregation Systems
  Recent advancements in deep learning have led to drastic improvements in
speech segregation models. Despite their success and growing applicability, few
efforts have been made to analyze the underlying principles that these networks
learn to perform segregation. Here we analyze the role of harmonicity on two
state-of-the-art Deep Neural Networks (DNN)-based models- Conv-TasNet and
DPT-Net. We evaluate their performance with mixtures of natural speech versus
slightly manipulated inharmonic speech, where harmonics are slightly frequency
jittered. We find that performance deteriorates significantly if one source is
even slightly harmonically jittered, e.g., an imperceptible 3% harmonic jitter
degrades performance of Conv-TasNet from 15.4 dB to 0.70 dB. Training the model
on inharmonic speech does not remedy this sensitivity, instead resulting in
worse performance on natural speech mixtures, making inharmonicity a powerful
adversarial factor in DNN models. Furthermore, additional analyses reveal that
DNN algorithms deviate markedly from biologically inspired algorithms that rely
primarily on timing cues and not harmonicity to segregate speech.
