When I set PP_size > 1 I get DP errpr for llama2 7b PEFT experiment using P-tuning
This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days. This issue was closed because it has been inactive for 7 days since being marked as stale.
**Describe the bug**

I have ran experiments for Multinode model customization (PEFT) using Nemo Framework training container. I tried both LoRa and Ptuning techniques, from which we are able to run LoRa successfully with various combination of TP and PP values. Although, with Ptuning as we increase PP more than 1, experiments fails with DDP error. Table below depicts what we tried in lab:

Error observed: RuntimeError: DistributedDataParallel is not needed when a module doesn't have any parameter that requires a gradient. 

Environment: 4 x Nodes (4 L40s each node) 

Model: llama2-7b

training container version: 23.11 

Table of experiments: 

LORA	TP	PP	Result
LORA	2	1	Success
LORA	4	1	Success
LORA	1	4	Success
LORA	4	4	Success
Ptuning	4	1	Success
Ptuning	1	2	Failed
Ptuning	1	4	Failed
Ptuning	2	2	Failed
Ptuning	2	4	Failed
Ptuning	4	2	Failed
Ptuning	4	4	Failed

Do you have any thoughts or suggestions on why p-tuning fails when PP size is higher than 1? 


**Expected behavior**

Customization job should proceed with requested configuration.

**Environment overview (please complete the following information)**

 - Environment location: Slurm batch using srun <pyxis>
 - Method of NeMo install: Container from NGC

**Environment details**
Environment: 4 x Nodes (4 L40s each node) 


