Speech Enhancement using Self-Adaptation and Multi-Head Self-Attention
  This paper investigates a self-adaptation method for speech enhancement using
auxiliary speaker-aware features; we extract a speaker representation used for
adaptation directly from the test utterance. Conventional studies of deep
neural network (DNN)--based speech enhancement mainly focus on building a
speaker independent model. Meanwhile, in speech applications including speech
recognition and synthesis, it is known that model adaptation to the target
speaker improves the accuracy. Our research question is whether a DNN for
speech enhancement can be adopted to unknown speakers without any auxiliary
guidance signal in test-phase. To achieve this, we adopt multi-task learning of
speech enhancement and speaker identification, and use the output of the final
hidden layer of speaker identification branch as an auxiliary feature. In
addition, we use multi-head self-attention for capturing long-term dependencies
in the speech and noise. Experimental results on a public dataset show that our
strategy achieves the state-of-the-art performance and also outperform
conventional methods in terms of subjective quality.
