Generation of Speaker Representations Using Heterogeneous Training Batch
  Assembly
  In traditional speaker diarization systems, a well-trained speaker model is a
key component to extract representations from consecutive and partially
overlapping segments in a long speech session. To be more consistent with the
back-end segmentation and clustering, we propose a new CNN-based speaker
modeling scheme, which takes into account the heterogeneity of the speakers in
each training segment and batch. We randomly and synthetically augment the
training data into a set of segments, each of which contains more than one
speaker and some overlapping parts. A soft label is imposed on each segment
based on its speaker occupation ratio, and the standard cross entropy loss is
implemented in model training. In this way, the speaker model should have the
ability to generate a geometrically meaningful embedding for each multi-speaker
segment. Experimental results show that our system is superior to the baseline
system using x-vectors in two speaker diarization tasks. In the CALLHOME task
trained on the NIST SRE and Switchboard datasets, our system achieves a
relative reduction of 12.93% in DER. In Track 2 of CHiME-6, our system provides
13.24%, 12.60%, and 5.65% relative reductions in DER, JER, and WER,
respectively.
