Questions regarding training conformer transducer model.
Hello,

Thank you for your work and all the help provided so far with other issues too. 

I am training a conformer transducer model and I have a couple of questions that might help me achieve better results:

1. I am finetuning the **stt en conformer transducer medium** model (whole model) on 225h of romanian language ( 200h training, 20h validation). I am at epoch 100 and the val_wer keeps decreasing, as well as the training wer. However, the performances on the eval set are still far away in performance comparing to our Kaldi model (10-15% more WER). How many epochs are usually necessary for convergence of the transducer medium model and are 200 hours of training data enough for a conformer model?

2. Do you use any type of cross-validation or simply randomly split the training set in train/val? 

3. At the moment I am using specaugment and spec cutout, should I also add noise impulses/background noise from some datasets too as augmentation? Could you please point what I should add in the config file exactly or if there is an example I could follow?

4. Lastly, I am using an LR of 1 with Noam annealing and 10% warm up ratio, is it a big LR for finetuning? ( from what I understood, you used 5 as LR) I am unsure if I should lower it to 0.1 maybe or 0.01.

Training takes quite a long time on our setup and we can't afford to test many configurations, that's why these questions may be pretty basic, but any insight is greatly appreciated!


