NameError: name 'ApexGuardDefaults' is not defined
is your issue solved ? if solved tell me how i am facing similar issue I came across this as well. @yonas-g if you found a solution would you mind sharing?

Could be missing an import `pip install megatron_core`
**NameError: name `'ApexGuardDefaults'` is not defined**

I am getting 
```bash
NameError: name 'ApexGuardDefaults' is not defined
```
when import `nemo_nlp` as `from nemo.collections import nlp as nemo_nlp`. 

I installed NeMo after cloning via `./reinstall.sh` command.

```bash
29 except (ImportError, ModuleNotFoundError) as e:
---> 31     TransformerLayer = TransformerLayerSubmodules = ApexGuardDefaults
     32     MLP = MLPSubmodules = ModuleSpec = IdentityOp = ApexGuardDefaults
     33     AttnMaskType = DotProductAttention = TENorm = ApexGuardDefaults

NameError: name 'ApexGuardDefaults' is not defined
```

**Environment overview (please complete the following information)**

 - Environment location:  AWS AMI, V100
 - Method of NeMo install: from source

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS: Ubuntu 20.04
- Torch: 2.3.0+cu121
- Python version: Python 3.11.9
