RuntimeError: world_size (1) is not divisible by 2
This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days. This issue was closed because it has been inactive for 7 days since being marked as stale.
**Describe the bug**

Unable to run the hello world script `nemo llm pretrain --factory llama3_8b` like in the documentation https://github.com/NVIDIA/NeMo/blob/main/examples/llm/pretrain/README.md

I get: 
```
[rank0]:   File "/opt/megatron-lm/megatron/core/parallel_state.py", line 532, in initialize_model_parallel
[rank0]:     raise RuntimeError(f"world_size ({world_size}) is not divisible by {total_model_size}")
[rank0]: RuntimeError: world_size (1) is not divisible by 2
```

**Steps/Code to reproduce bug**

`srun -p batch -N 1 --container-image nvcr.io/nvidia/nemo:24.09 --pty bash`
The machine has 8 gpu according to nvidia-smi 
`run nemo llm pretrain --factory llama3_8b`

**Expected behavior**

I would expect it to run seamlessly 

**Environment overview (please complete the following information)**

`srun -p batch -N 1 --container-image nvcr.io/nvidia/nemo:24.09 --pty bash`
The machine has 8 gpu according to nvidia-smi 
