100% WER on twice fine-Tuning with Persian audio using NEMO QuartzNet15*5
Does the loss reduce to earlier levels when you combine datasets? How many finetuning steps are performed? > Does the loss reduce to earlier levels when you combine datasets? How many finetuning steps are performed?

Yes. The loss reduces. I test different epochs such as 5,10,20,50,100, and 200 but the results are the same. It is hard to debug without the dataset. The fact that loss reduces properly means the model is training - however it cannot be known why predictions are not correct at test time. I would suggest training with just the smaller dataset first to see if it learns at all - long clips might simply be causing wer to jump since model is unable to learn both short and long audio.

Also can you try combining the dataset and training from scratch if that works 
**Fine-tuning on restored checkpoint has WER 100%  with Persian audio**
Hi 
I fine-tuned the Quartz Net 15*5 model with 235 hours dataset of Persian Mozilla Common Voice. When I tested the fine-tuned model with Persian Common Voice It worked well with WER 35%. So I saved the checkpoint of it.
Now I restore that checkpoint. I start fine-tune it with another Persian dataset in new domain. The audios of new dataset are recorded telephony audio. I changed the sample rate of new dataset from 8KHZ to 16KHZ and convert each telephony audio to different chunks due to long duration and lack of RAM problem in Colab tarring process.
After I fine-tuned the checkpoint  with new dataset I test it but the test result in transcription is not good any more. The model can not transcribe any audio even one correct word. In fact it transcribe some alphabet not a complete word or sentence.
What`s your suggestion for new fine-tuning with telephony Persian audio dataset on the mentioned checkpoint?

**Environment overview**

- Environment location: Google Colab

- Method of NeMo install: !pip install nemo_toolkit[asr]

- NeMo version: 1.0.0

- Learning Rate: 1e-5


**Environment details**

- OS version : "Ubuntu20.04.3 LTS"
- PyTorch version : "1.7.1"
- Dataset used for first Fine-Tuning: "Persian Common Voice" (Mozilla) and duration: 235 hours
- Dataset used for first Fine-Tuning: "Persian Telephony Audio" and duration: 22 minutes