Use the eval_beamsearch_ngram.py and add the LM made the CER worse.
**Describe the bug**
Hi 

I found that using eval_beamsearch_ngram.py to evaluate the language model (LM) could reduce the CER by how much, but I actually found that adding the LM made it worse.

I used pretrain am model=" Conformer-CTC-L-unified_char_ja-JP_6.0.nemo"
and pretrain LM model ="riva_zh_asr_set_2.1_4gram.bin"
my testing data is used commo

I use decoding_mode=greedy and get 
`[NeMo I 2023-09-21 15:44:32 eval_beamsearch_ngram:372] Greedy WER/CER = 43.38%/10.48%
`
but when i use 
decoding_mode=beamsearch_ngram 
decoding_strategy="flashlight"
 
`[NeMo I 2023-09-21 15:46:57 eval_beamsearch_ngram:237] WER/CER with beam search decoding and N-gram model = 99.80%/24.11%
[NeMo I 2023-09-21 15:46:57 eval_beamsearch_ngram:248] Oracle WER/CER in candidates with perfect LM= 43.36%/10.72%
[NeMo I 2023-09-21 15:46:57 eval_beamsearch_ngram:253] =================================================================================
[NeMo I 2023-09-21 15:46:57 eval_beamsearch_ngram:439] Best WER Candidate = 99.80% :: Beam size = 64, Beam alpha = 1.0, Beam beta = 0.0
[NeMo I 2023-09-21 15:46:57 eval_beamsearch_ngram:444] Best CER Candidate = 24.11% :: Beam size = 64, Beam alpha = 1.0, Beam beta = 0.0
[NeMo I 2023-09-21 15:46:57 eval_beamsearch_ngram:448] =================================================================================`

CER got worse.

**Describe the solution you'd like**
Here is the code I am using

and I found that after each sentence in preds_output_folder, an unrelated English word is appended.

外交与全球性议题E	-40.78262926785392
外交与全球性议题L	-41.35890473572654
外交与全球性议题I	-41.67608906475944
外交与全球性议题A	-41.78252627102775
外交与全球性议题D	-41.92528642861289
外交与全球性议题J	-42.08723165242118
外交与全球性议题K	-42.095088259858315


**Steps/Code to reproduce bug**

```
python eval_beamsearch_ngram.py nemo_model_file=/model/zh-TW/stt_zh_citrinet_1024_gamma_0_25.nemo \
       input_manifest=/manifests/zh-TW/commonvoice_test_manifest_processed.json \
       kenlm_model_file=/model/zh-TW/lm/riva_zh_asr_set_2.1_4gram.bin \
       preds_output_folder=./ \
       probs_cache_file=null \
       beam_width=[20] \
       beam_alpha=[1.0] \
       beam_beta=[1] \
       decoding_mode=beamsearch_ngram \
       decoding_strategy="flashlight"
```

**Expected behavior**

How to reduce CER while increasing the LM?

