Support for per-dataset metrics
This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days. This issue was closed because it has been inactive for 7 days since being marked as stale.
**Is your feature request related to a problem? Please describe.**
I'd like to be able to get per-dataset metrics during training & eval(testing). 
Currently even if I configure the `model.data` with multiple datasets, the val_loss still only shows the total loss not the per-dataset losses. See attached.
Looking at the code, it looks like this is due to NeMo blending the datasets before feeding to the PL trainer (to be confirmed by NeMo engineers). [Code reference](https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/nlp/models/language_modeling/megatron_gpt_model.py).

![image](https://github.com/user-attachments/assets/08d3537e-6cc4-466d-8309-efd99f61e23a)


**Describe the solution you'd like**
I would like that NeMo would have the option to enable per-dataset metrics. So that for each val loop, a set of losses (each corresponding to an input dataset) would be logged.

This is hardly a technical proposal but my intuition is that we could either make it so that one dataloader is created per dataset (this might require not blending the datasets in the first place) and pass a map/list of dataloaders to the PL trainer so that the default behavior of the trainer can be leveraged. Or we could take advantage of the `dataset_id` [field](https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/datasets/blended_dataset.py#L96) from each datum and have some sort of breakdown mechanism in the metrics calculation stage. 

**Describe alternatives you've considered**
N/A

**Additional context**
N/A
