Draft of fix for #7428
# What does this PR do ?

Draft fix for #7428

**Collection**: [nlp]

# Changelog 
- Fix handling of `limit_val_batches` which could lead to crashes or computing validation error on less data than intended

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Fixes #7428

This is a draft as I am unsure this is the "right" way to fix it, and more work is needed to finish it. However, I would like to request feedback from NeMo devs to know whether this is heading in the right direction. Some things to think about:
* Probably that other classes need to be modified as well, beyond MegatronGPTModel
* Changing the length of the validation dataloader might have unintended consequences
* The MegatronPretrainingRandomSampler might need some fix as well
* I think (at first glance) that the dataloaders are not properly implementing `drop_last` since it seems to be based on `micro_batch_times_data_parallel_size` rather than the global batch size (I didn't try to fix it in this PR)
* I haven't tested at all my fix for the float `limit_val_batches`
* Does the test set also need something similar?
* ... (this is definitely rushed => there's a good chance I may have missed more stuff)