Listening to Sounds of Silence for Speech Denoising
  We introduce a deep learning model for speech denoising, a long-standing
challenge in audio analysis arising in numerous applications. Our approach is
based on a key observation about human speech: there is often a short pause
between each sentence or word. In a recorded speech signal, those pauses
introduce a series of time periods during which only noise is present. We
leverage these incidental silent intervals to learn a model for automatic
speech denoising given only mono-channel audio. Detected silent intervals over
time expose not just pure noise but its time-varying features, allowing the
model to learn noise dynamics and suppress it from the speech signal.
Experiments on multiple datasets confirm the pivotal role of silent interval
detection for speech denoising, and our method outperforms several
state-of-the-art denoising methods, including those that accept only audio
input (like ours) and those that denoise based on audiovisual input (and hence
require more information). We also show that our method enjoys excellent
generalization properties, such as denoising spoken languages not seen during
training.
