Fine-tune stt_zh_quartznet15x5 model  with custom dataset
**Describe the bug**

1. Output reference is not correct on chinese, could it affect the model?
![image](https://user-images.githubusercontent.com/25794106/130772485-45c4e6c0-470c-431f-8230-1ef2a296dc23.png)

  I follow the  link below:
  https://colab.research.google.com/github/NVIDIA/NeMo/blob/stable/tutorials/asr/10_ASR_CTC_Language_Finetuning.ipynb#scrollTo=x0i8hvt688hc
2. I have four gpus, But  just use one; 





**Steps/Code to reproduce bug**

 step 1: I  prepare the dataset ,  
          **manifest files:**
          train_manifest="dataset/manifest_train.json"
          test_manifest = "dataset/manifest_test.json"
          dev_manifest = "dataset/manifest_dev.json"
         **charset:**
          train_charset = get_charset(train_manifest_data)
          dev_charset = get_charset(dev_manifest_data)
          test_charset = get_charset(test_manifest_data)

      
step 2: load model config , update the config 
            freeze_encoder = True
            cfg.train_ds.manifest_filepath  = "train_manifest,dev_manifest"

            use_cer = True
            log_prediction = True
            model._wer.use_cer = use_cer
            model._wer.log_prediction = log_prediction

step 3: train 
          
![image](https://user-images.githubusercontent.com/25794106/130769707-8ac3b2d4-dc08-4878-9d1c-8c4a907f34ba.png)

         
**Expected behavior**

the reference part  should be the correct sentence；
as you see ,the output is out-of-order ， I expect the out  is  1 2 3 4 ,but the result is 2 1 3 4 ;
**Environment overview (please complete the following information)**

 - Environment location: local server  DGX V100
 - Method of NeMo install:  pip install 


**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version :ubuntu 18.04
- PyTorch version:pytorch 1.9.0
- Python version :python3.7

**Additional context**

This is my train script:
[nemo_train.txt](https://github.com/NVIDIA/NeMo/files/7046204/nemo_train.txt)

