VoiceFilter-Lite: Streaming Targeted Voice Separation for On-Device
  Speech Recognition
  We introduce VoiceFilter-Lite, a single-channel source separation model that
runs on the device to preserve only the speech signals from a target user, as
part of a streaming speech recognition system. Delivering such a model presents
numerous challenges: It should improve the performance when the input signal
consists of overlapped speech, and must not hurt the speech recognition
performance under all other acoustic conditions. Besides, this model must be
tiny, fast, and perform inference in a streaming fashion, in order to have
minimal impact on CPU, memory, battery and latency. We propose novel techniques
to meet these multi-faceted requirements, including using a new asymmetric
loss, and adopting adaptive runtime suppression strength. We also show that
such a model can be quantized as a 8-bit integer model and run in realtime.
