The 2021 NIST Speaker Recognition Evaluation
  The 2021 Speaker Recognition Evaluation (SRE21) was the latest cycle of the
ongoing evaluation series conducted by the U.S. National Institute of Standards
and Technology (NIST) since 1996. It was the second large-scale multimodal
speaker/person recognition evaluation organized by NIST (the first one being
SRE19). Similar to SRE19, it featured two core evaluation tracks, namely audio
and audio-visual, as well as an optional visual track. In addition to offering
fixed and open training conditions, it also introduced new challenges for the
community, thanks to a new multimodal (i.e., audio, video, and selfie images)
and multilingual (i.e., with multilingual speakers) corpus, termed WeCanTalk,
collected outside North America by the Linguistic Data Consortium (LDC). These
challenges included: 1) trials (target and non-target) with enrollment and test
segments originating from different domains (i.e., telephony versus video), and
2) trials (target and non-target) with enrollment and test segments spoken in
different languages (i.e., cross-lingual trials). This paper presents an
overview of SRE21 including the tasks, performance metric, data, evaluation
protocol, results and system performance analyses. A total of 23 organizations
(forming 15 teams) from academia and industry participated in SRE21 and
submitted 158 valid system outputs. Evaluation results indicate: audio-visual
fusion produce substantial gains in performance over audio-only or visual-only
systems; top performing speaker and face recognition systems exhibited
comparable performance under the matched domain conditions present in this
evaluation; and, the use of complex neural network architectures (e.g., ResNet)
along with angular losses with margin, data augmentation, as well as long
duration fine-tuning contributed to notable performance improvements for the
audio-only speaker recognition task.
