Temporal-Spatial Neural Filter: Direction Informed End-to-End
  Multi-channel Target Speech Separation
  Target speech separation refers to extracting the target speaker's speech
from mixed signals. Despite the recent advances in deep learning based
close-talk speech separation, the applications to real-world are still an open
issue. Two main challenges are the complex acoustic environment and the
real-time processing requirement. To address these challenges, we propose a
temporal-spatial neural filter, which directly estimates the target speech
waveform from multi-speaker mixture in reverberant environments, assisted with
directional information of the speaker(s). Firstly, against variations brought
by complex environment, the key idea is to increase the acoustic representation
completeness through the jointly modeling of temporal, spectral and spatial
discriminability between the target and interference source. Specifically,
temporal, spectral, spatial along with the designed directional features are
integrated to create a joint acoustic representation. Secondly, to reduce the
latency, we design a fully-convolutional autoencoder framework, which is purely
end-to-end and single-pass. All the feature computation is implemented by the
network layers and operations to speed up the separation procedure. Evaluation
is conducted on simulated reverberant dataset WSJ0-2mix and WSJ0-3mix under
speaker-independent scenario. Experimental results demonstrate that the
proposed method outperforms state-of-the-art deep learning based multi-channel
approaches with fewer parameters and faster processing speed. Furthermore, the
proposed temporal-spatial neural filter can handle mixtures with varying and
unknown number of speakers and exhibits persistent performance even when
existing a direction estimation error. Codes and models will be released soon.
