The loss of `MegatronGPTModel` is incorrect for `micro_batch_size` > 1
This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days. This issue was closed because it has been inactive for 7 days since being marked as stale.
**Describe the bug**

The loss implemented [here](https://github.com/NVIDIA/NeMo/blob/13c1db4ffdaa3b2125ce6e66c93509a39d55b2cf/nemo/collections/nlp/models/language_modeling/megatron_gpt_model.py#L1169) is incorrect for `micro_batch_size` > 1 because it computes the average over all tokens in the micro-batch, instead of computing a per-sample loss then averaging it. This leads to inconsistent results when varying `micro_batch_size`.

**Steps/Code to reproduce bug**

Compare results with different values of `micro_batch_size` and notice they are very different.

**Expected behavior**

Varying `micro_batch_size` shouldn't affect results.

**Additional context**

I am creating this issue for visibility (in case someone else runs into the same problem), and will be submitting a PR to fix it.