How to choose number of epochs in Conformer-Transducer (RNNT) pretrained model finetuning
Hi All,

I have taken conformer-transducer (RNNT) & conformer CTC pretrained English medium models, and finetuned them for Tamil dataset. Finetuned CTC has decent WERs, whereas RNNT models have unacceptable WERs. Both were trained for 18 epochs. The CTC finetuning has consistently exponentially decreasing Validation loss. However, validation loss of RNNT consistently increased after 5th epoch. PFA RNNT val loss snapshot.

![image](https://user-images.githubusercontent.com/6142443/212663699-62081f70-cf12-418d-9e4d-52b1464aa57b.png)

So, in this case, 
Question 1: Do you suggest me to train only for 5 epochs while using RNNT and use the saved model for decoding?
Question 2: Is there any way like any parameter to be set either in exp_manager or trainer, so that it would automatically save the check point with best WER
Question 3: Am I doing anything wrong here. Should I need to check/change anything to make RNNT validation error to decrease.

Note: We have done several experiments with Conformer-CTC and they seem to work without any issues with same setup (like no. of epochs etc). However, we are exploring the RNNT for the first time. So, not sure if something is missing.
Please suggest @titu1994  Thanks. 