[Question] How to perform generation with the GPT model checkpoints trained with virtual staging
I have a question on how we can load the GPT model checkpoints trained with virtual staging and perform generation? With virtual staging in pipeline parallelism, each rank stores a list of modules. But I cannot find relevant code for dealing with such case when performing generation. It seems to me NeMo does not support inference when the pretrained model is trained with virtual staging.