Fix Canary not stripping prompt from reference + more test coverage
> One name change, fix import flags from the CI.

OK

> Btw, how are you feeding temperature to the MLP now?

It's applied directly in `Categorical` 
# What does this PR do ?

TL;DR: fix for a bug I accidentally introduced in #9901 

Context:
* #9901 changed multi-task beam search decoding module to strip the prompt and EOS away form recognitions; it removed "remove_special_tokens" sprinkled in various places in our codebase (transcribe, wer, bleu, etc.)
* That fix was actually porting a part of my changes in an old closed PR #8833. However, I forgot that we also have to handle the dataset part properly - return GT transcripts with stripped prompts and EOS (which was done there). So I ended up introduced a bug that reference has the prompt, but recognition doesn't in scoring.
* I ported the necessary additional changes from #8833 and adapted them to the current codebase.
* Since I wanted to improve the test coverage of the affected parts, I also pulled the changes from #8885 (another PR I forgot to merge) where I already wrote these tests, and extended them.

**Collection**: ASR

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
