"Loop labels" greedy decoding: faster implementation
jenkins jenkins jenkins
# What does this PR do ?

Improves "loop labels" greedy decoding implementation, introduced in #7926, with using non-blocking operations (avoiding device-to-host synchronization).

On my local machine, the time for decoding unsorted `test-other` (LibriSpeech) with `stt_en_fastconformer_transducer_large` (2 repeated measurements, in seconds; after warmup the results are better):

 bs | precision  | RNN-T, loop_labels v1 (main) | RNN-T, loop_labels v2 (this)
-- | -- | -- | --
16 | fp32 | 50s / 50s | 37s / 36s
128 | fp32 | 34s / 33s | 30s / 29s
16 | bf16 | 48s / 46s | 34s / 32s
128 | bf16 | 25s / 24s | 21s / 20s

This algorithm is planned for further improvements with torch.jit.script in #8203

**Collection**: [ASR]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage

```shell
python examples/asr/speech_to_text_eval.py \
   model_path=<nemo_model.nemo> \
   dataset_manifest=<manifest> \
   batch_size=128 \
   output_filename=<output_mainfest_path> \
   rnnt_decoding.greedy.loop_labels=true
```

# Jenkins CI
To run Jenkins, a NeMo User with write access must comment `jenkins` on the PR.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
