Support BLOOM for finetuning?
I noticed that you referenced the ALiBi in the flash-attn module, but I couldn't find any mention or description of its usage in your project. I'm curious to know if you have incorporated support for models like BLOOM for fine-tuning and converting their checkpoints from Huggingface format to .nemo or MegatronLM.