Loading 70B model from .nemo checkpoint takes very long time
This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days. This issue was closed because it has been inactive for 7 days since being marked as stale.
I am launching a continuous pretraining job from a 70B model checkpoint. The checkpoint is saved as .nemo file. I use
```
megatronGPTModel.restore_from(restore_path="/path/to/my/70B/.nemo", override_config_path, trainer)
```
to restore the checkpoint. It hangs while unpacking the .nemo file.

The container I am using is `nvcr.io/nvidia/nemo:24.05.llama3.1`