End-to-end Microphone Permutation and Number Invariant Multi-channel
  Speech Separation
  An important problem in ad-hoc microphone speech separation is how to
guarantee the robustness of a system with respect to the locations and numbers
of microphones. The former requires the system to be invariant to different
indexing of the microphones with the same locations, while the latter requires
the system to be able to process inputs with varying dimensions. Conventional
optimization-based beamforming techniques satisfy these requirements by
definition, while for deep learning-based end-to-end systems those constraints
are not fully addressed. In this paper, we propose
transform-average-concatenate (TAC), a simple design paradigm for channel
permutation and number invariant multi-channel speech separation. Based on the
filter-and-sum network (FaSNet), a recently proposed end-to-end time-domain
beamforming system, we show how TAC significantly improves the separation
performance across various numbers of microphones in noisy reverberant
separation tasks with ad-hoc arrays. Moreover, we show that TAC also
significantly improves the separation performance with fixed geometry array
configuration, further proving the effectiveness of the proposed paradigm in
the general problem of multi-microphone speech separation.
