How to reproduce the throughput in the blog
I managed to reproduce🙃
**Describe the bug**
Refer to this blog, training LLaMA 13B with NeMo 23.08 could achieve 2357 Tokens/s/GPU
![image](https://github.com/NVIDIA/NeMo/assets/73142299/da55a7eb-0992-440c-9c32-4aba21150d99)

Yet, I can only achieve under 1900 Tokens/s/GPU
![image](https://github.com/NVIDIA/NeMo/assets/73142299/4f5a5626-2e8b-42aa-a4f3-f14ba4973d20)


**Steps/Code to reproduce bug**
I'm running on nvcr.io/nvidia/nemo:23.08 with 8xA100
Here is my launching script
```
INDEX_MAPPING_DIR=""
DATA_PREFIX=""
TOKENIZER_DIR=""
TP_SIZE=4
PP_SIZE=1

torchrun --nproc_per_node=8 \
/workspace/nemo/examples/nlp/language_modeling/megatron_llama_pretraining.py \
   trainer.precision=bf16 \
   trainer.devices=8 \
   trainer.num_nodes=1 \
   model.num_layers=40 \
   model.hidden_size=5120 \
   model.ffn_hidden_size=13824 \
   model.num_attention_heads=40 \
   model.num_query_groups=40 \
   model.micro_batch_size=1 \
   model.global_batch_size=32 \
   model.tensor_model_parallel_size=${TP_SIZE} \
   model.pipeline_model_parallel_size=${PP_SIZE} \
   model.megatron_amp_O2=False \
   model.sequence_parallel=True \
   model.activations_checkpoint_granularity=selective \
   model.activations_checkpoint_method=uniform \
   model.optim.name=distributed_fused_adam \
   model.optim.lr=2e-4 \
   model.tokenizer.model=${TOKENIZER_DIR} \
   model.data.index_mapping_dir=${INDEX_MAPPING_DIR} \
   ++model.data.data_prefix=${DATA_PREFIX} \
   exp_manager.create_wandb_logger=False \
   exp_manager.explicit_log_dir=/results \
   exp_manager.resume_if_exists=False \
   exp_manager.resume_ignore_no_checkpoint=True \
   exp_manager.create_checkpoint_callback=True \
   exp_manager.checkpoint_callback_params.monitor=validation_loss \
   ++cluster_type=BCP
```
The `megatron_llama_pretraining.py`
```
import torch.multiprocessing as mp
from omegaconf.omegaconf import OmegaConf, open_dict

from nemo.collections.nlp.models.language_modeling.megatron_gpt_model import MegatronGPTModel
from nemo.collections.nlp.parts.megatron_trainer_builder import MegatronTrainerBuilder
from nemo.core.config import hydra_runner
from nemo.utils import logging
from nemo.utils.exp_manager import exp_manager

mp.set_start_method("spawn", force=True)


@hydra_runner(config_path="conf", config_name="megatron_llama_config")
def main(cfg) -> None:
    logging.info("\n\n************** Experiment configuration ***********")
    logging.info(f'\n{OmegaConf.to_yaml(cfg)}')

    trainer = MegatronTrainerBuilder(cfg).create_trainer()
    exp_manager(trainer, cfg.exp_manager)

    model = MegatronGPTModel(cfg.model, trainer)

    trainer.fit(model)


if __name__ == '__main__':
    main()
```


**Expected behavior**
Do I have to modify the configuration to get better performance? If so, could you show the config🥺?
