Training Speech Enhancement Systems with Noisy Speech Datasets
  Recently, deep neural network (DNN)-based speech enhancement (SE) systems
have been used with great success. During training, such systems require clean
speech data - ideally, in large quantity with a variety of acoustic conditions,
many different speaker characteristics and for a given sampling rate (e.g.,
48kHz for fullband SE). However, obtaining such clean speech data is not
straightforward - especially, if only considering publicly available datasets.
At the same time, a lot of material for automatic speech recognition (ASR) with
the desired acoustic/speaker/sampling rate characteristics is publicly
available except being clean, i.e., it also contains background noise as this
is even often desired in order to have ASR systems that are noise-robust.
Hence, using such data to train SE systems is not straightforward. In this
paper, we propose two improvements to train SE systems on noisy speech data.
First, we propose several modifications of the loss functions, which make them
robust against noisy speech targets. In particular, computing the median over
the sample axis before averaging over time-frequency bins allows to use such
data. Furthermore, we propose a noise augmentation scheme for mixture-invariant
training (MixIT), which allows using it also in such scenarios. For our
experiments, we use the Mozilla Common Voice dataset and we show that using our
robust loss function improves PESQ by up to 0.19 compared to a system trained
in the traditional way. Similarly, for MixIT we can see an improvement of up to
0.27 in PESQ when using our proposed noise augmentation.
