STC speaker recognition systems for the NIST SRE 2021
  This paper presents a description of STC Ltd. systems submitted to the NIST
2021 Speaker Recognition Evaluation for both fixed and open training
conditions. These systems consists of a number of diverse subsystems based on
using deep neural networks as feature extractors. During the NIST 2021 SRE
challenge we focused on the training of the state-of-the-art deep speaker
embeddings extractors like ResNets and ECAPA networks by using additive angular
margin based loss functions. Additionally, inspired by the recent success of
the wav2vec 2.0 features in automatic speech recognition we explored the
effectiveness of this approach for the speaker verification filed. According to
our observation the fine-tuning of the pretrained large wav2vec 2.0 model
provides our best performing systems for open track condition. Our experiments
with wav2vec 2.0 based extractors for the fixed condition showed that
unsupervised autoregressive pretraining with Contrastive Predictive Coding loss
opens the door to training powerful transformer-based extractors from raw
speech signals. For video modality we developed our best solution with
RetinaFace face detector and deep ResNet face embeddings extractor trained on
large face image datasets. The final results for primary systems were obtained
by different configurations of subsystems fusion on the score level followed by
score calibration.
