Converting HF model to Nemo gets an error
Following up regarding issue. Please let me know if any more information is needed. According to this [HF implementation](https://github.com/huggingface/transformers/blob/v4.44.2/src/transformers/models/opt/modeling_opt.py):
* `project_out` should be the final output layer, so it should be mapped to `model.output_layer.weight`
* `project_in` seems to be a linear layer projecting from embedding dim down to hidden size. Not sure if NeMo supports it. This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days. @qijiaxing 
isn't the 'lm_head.weight' key should be mapped to model.output_layer.weight?
To give more understanding, I mapped lm_head.weight to model.output_layer.weight, which seems to be working fine. 
now I am left with mapping
HF keys : ['model.decoder.project_out.weight', 'model.decoder.project_in.weight'] to
Nemo Keys.

Surprising it complains ['model.decoder.final_layernorm.weight', 'model.decoder.final_layernorm.bias'] from Nemo side, even though I made sure it doesn't include final layer norm explicitly.

Please let me know if needed any more explaination. This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days. This issue was closed because it has been inactive for 7 days since being marked as stale.
**Describe the bug**
I am trying to convert the [OPT-350m](https://huggingface.co/facebook/opt-350m) hugging face model to Nemo format. How can I map the following keys?
```
- model.decoder.project_out.weight
- model.decoder.project_in.weight
```
for other weight names, I can successfully map to Nemo format.

A clear and concise description of what the bug is.

**Steps/Code to reproduce bug**

Please list *minimal* steps or code snippet for us to be able to reproduce the bug.

A  helpful guide on on how to craft a minimal bug report  http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports. 


**Expected behavior**

A clear and concise description of what you expected to happen.

**Environment overview (please complete the following information)**

 - Environment location: [Bare-metal, Docker, Cloud(specify cloud provider - AWS, Azure, GCP, Collab)] Docker
 - Method of NeMo install: [pip install or from source]. Please specify exact commands you used to install. nemo:24.03.01.framework
 - If method of install is [Docker], provide `docker pull` & `docker run` commands used

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version : Ubuntu 20.04
- PyTorch version : nemo:24.03.01.framework
- Python version : nemo:24.03.01.framework

**Additional context**

Add any other context about the problem here.
Example: GPU model : H100
