Error running Speaker Recognition notebook: arguments are located on different GPUs
I am also getting the same issue.
https://github.com/NVIDIA/NeMo/issues/1502 this might help. please use `ddp` instead of `dp` as accelerator. Feel free to open the issue again incase you encounter it again. 
**Describe the bug**

When running a speaker recognition notebook on multi GPUs machine I get the following error:
RuntimeError: arguments are located on different GPUs at /tmp/pip-req-build-xlj_h8ax/aten/src/THC/generic/THCTensorMathBlas.cu:25

**Steps/Code to reproduce bug**

1) 8-V100 GPUs machine
2) using the latests NeMo docker image 1.0.0b3.
3) run the Speaker_Recognition_Verification.ipynb notebook
4) modified the trainer sets up cell to 
```
  # Let us modify some trainer configs for this demo
  # Checks if we have GPU available and uses it
  cuda = 1 if torch.cuda.is_available() else 0
  config.trainer.gpus = 4

  # Reduces maximum number of epochs to 5 for quick demonstration
  config.trainer.max_epochs = 5

  # Remove distributed training flags
  config.trainer.accelerator = 'dp'
 ```



**Expected behavior**
running the trainer.fit(speaker_model) I get:

```
...

    357         # dot with filterbank energies
--> 358         x = torch.matmul(self.fb.to(x.dtype), x)
    359 
    360         # log features if required

RuntimeError: arguments are located on different GPUs at /tmp/pip-req-build-xlj_h8ax/aten/src/THC/generic/THCTensorMathBlas.cu:25

```
**Environment overview (please complete the following information)**

docker run command used:
```
docker run --gpus all -it --rm  -d -v `pwd`:/NeMo -v /home/nshaposhnikov/datasets:/datasets -v /home/nshaposhnikov/work:/work --shm-size=8g -p 8888:8888 -p 6006:6006 --ulimit memlock=-1 --ulimit stack=67108864 --device=/dev/snd nvcr.io/nvidia/nemo:1.0.0b3
Unable to find image 'nvcr.io/nvidia/nemo:1.0.0b3' locally
1.0.0b3: Pulling from nvidia/nemo
f08d8e2a3ba1: Already exists 
3baa9cb2483b: Already exists 
```
