Continuous Speech Separation with Ad Hoc Microphone Arrays
  Speech separation has been shown effective for multi-talker speech
recognition. Under the ad hoc microphone array setup where the array consists
of spatially distributed asynchronous microphones, additional challenges must
be overcome as the geometry and number of microphones are unknown beforehand.
Prior studies show, with a spatial-temporalinterleaving structure, neural
networks can efficiently utilize the multi-channel signals of the ad hoc array.
In this paper, we further extend this approach to continuous speech separation.
Several techniques are introduced to enable speech separation for real
continuous recordings. First, we apply a transformer-based network for
spatio-temporal modeling of the ad hoc array signals. In addition, two methods
are proposed to mitigate a speech duplication problem during single talker
segments, which seems more severe in the ad hoc array scenarios. One method is
device distortion simulation for reducing the acoustic mismatch between
simulated training data and real recordings. The other is speaker counting to
detect the single speaker segments and merge the output signal channels.
Experimental results for AdHoc-LibiCSS, a new dataset consisting of continuous
recordings of concatenated LibriSpeech utterances obtained by multiple
different devices, show the proposed separation method can significantly
improve the ASR accuracy for overlapped speech with little performance
degradation for single talker segments.
