List index out of range when transcribing with RNNT conformer large when batch size > 1
**Describe the bug**

Hello, although #7192 fixed most of our errors, we are still running into some specific cases where the model still fails with the following stack trace.
![image](https://github.com/NVIDIA/NeMo/assets/10922796/db248328-3490-43e6-9f3a-fa667f955f57)

I also noticed that this error only happens when the batch size is higher than 1, while the model transcribes fine with batch_size 1, although of course being slower with a lot of audio files.
The weird part is that this only seems to happen with a combination of certain audio files, as in transcribing 1 of the 2 files 20 times doesnt give an error.

**Steps/Code to reproduce bug**

Luckily I was able to reproduce this error in the English model.

I used a new docker ( nvcr.io/nvidia/pytorch:23.04-py3 ).

apt-get update && apt-get install -y libsndfile1 ffmpeg
git clone https://github.com/NVIDIA/NeMo
cd NeMo
./reinstall.sh

The 2 audio files can be downloaded here: https://easyupload.io/m/980zua

Next I ran the following script:

import nemo
import nemo.collections.asr as nemo_asr
import copy
from omegaconf import DictConfig, OmegaConf, open_dict
from nemo.collections.asr.parts.utils.asr_confidence_utils import ConfidenceConfig

conformer = nemo_asr.models.EncDecRNNTBPEModel.from_pretrained(model_name="stt_en_conformer_transducer_large")

decoding_config = copy.deepcopy(conformer.cfg.decoding)
confidence_cfg = ConfidenceConfig(preserve_word_confidence=True, preserve_token_confidence=False)

with open_dict(decoding_config):
    decoding_config.preserve_alignments = True,
    decoding_config.confidence_cfg = confidence_cfg

conformer.change_decoding_strategy(decoding_config)

audio_segment_name_list = [ '/workspace/FailingCuts/12.wav', '/workspace/FailingCuts/13.wav']
hypotheses = conformer.transcribe(paths2audio_files=audio_segment_name_list, return_hypotheses=True, batch_size=2)

**Expected behavior**

Transcription succeeding without errors when using a batch size higher than 1

**Additional context**

Example: GPU model

Ran on a 4090 GPU.

@GNroy, would you be able to look at this again?
