Speaker Diarization takes too long
I have finetuned a speaker recognition model with the help of this tutorial : https://github.com/NVIDIA/NeMo/blob/main/tutorials/speaker_tasks/Speaker_Identification_Verification.ipynb
After that I trained a MSDD Model for Speaker Diarization and replaced the pretrained titanet model with my finetuned Speaker Recognition Model. I followed this tutorial :https://github.com/NVIDIA/NeMo/blob/main/tutorials/speaker_tasks/Speaker_Diarization_Training.ipynb
I tried both, training with a frozen speaker embeddings model and joint training and noticed that the validation loss was much lower when using joint training. Nevertheless, the inference time for Speaker Diarization takes forever when extracting embeddings. How can I tackle that problem?