End-to-End Diarization for Variable Number of Speakers with Local-Global
  Networks and Discriminative Speaker Embeddings
  We present an end-to-end deep network model that performs meeting diarization
from single-channel audio recordings. End-to-end diarization models have the
advantage of handling speaker overlap and enabling straightforward handling of
discriminative training, unlike traditional clustering-based diarization
methods. The proposed system is designed to handle meetings with unknown
numbers of speakers, using variable-number permutation-invariant cross-entropy
based loss functions. We introduce several components that appear to help with
diarization performance, including a local convolutional network followed by a
global self-attention module, multi-task transfer learning using a speaker
identification component, and a sequential approach where the model is refined
with a second stage. These are trained and validated on simulated meeting data
based on LibriSpeech and LibriTTS datasets; final evaluations are done using
LibriCSS, which consists of simulated meetings recorded using real acoustics
via loudspeaker playback. The proposed model performs better than previously
proposed end-to-end diarization models on these data.
