[Question] Pipeline Parallel for Mamba Megatron
@JRD971000 
Hi, 

I am wondering if the scripts implemented in Mamba Megatron pipeline parallel will be possibly used inside Nemo? Specifically Mamba Megatron supports converting Mamba both to Tensor Parallel & Pipeline Parallel [at here](https://github.com/NVIDIA/Megatron-LM/blob/main/tools/checkpoint/hybrid_conversion.py). The pipeline parallel code has been using Megatron's [without interleaved implementation](https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/pipeline_parallel/schedules.py#L1274). Will we be able to utilize any of that Megatron implementation for SFT on Mamba in Nemo? 

Thank you! 