List index out of range when transcribing with RNNT conformer large
**Describe the bug**

A clear and concise description of what the bug is.

Hello, we are running a Dutch trained version of the conformer RNNT large model.
However, in a small amount of cases we receive the following list index out of range error.

![image](https://github.com/NVIDIA/NeMo/assets/10922796/5341394d-a158-43d5-a724-52261052f0bb)

I saw that in [https://github.com/NVIDIA/NeMo/issues/4981 ](url) the issue seemed solved for the word timings, but maybe this issue still exists for the confidence scores?

We use the following to enable the confidence scores:

`
decoding_config = copy.deepcopy(conformer.cfg.decoding)
`
`
confidence_cfg = ConfidenceConfig(preserve_word_confidence=True, preserve_token_confidence=False)
`
`
with open_dict(decoding_config):
`
`
decoding_config.preserve_alignments = True,
`
`
    decoding_config.confidence_cfg = confidence_cfg
`
`
conformer.change_decoding_strategy(decoding_config)
`

**Expected behavior**

Transcription succeeding without errors

**Environment overview (please complete the following information)**

Docker nvcr.io/nvidia/pytorch:23.04-py3
NeMo installed from main with git clone and ./reinstall.sh ( cloned today )


**Additional context**

Ran on a 4090 GPU
