Error selecting speaker while inferencing
Hi all.

I've trained FastPitch using both LJSpeech and my dataset, as it is suggested in the FastPitch_finetuning.ipynb colab. I've balanced the datasets by time instead of by number of samples. It turns out that the voice generated is from the woman of LJSpeech, not from my dataset.

While trying to execute the inference, with the function `str_to_audio()`:

```
def str_to_audio(inp, pace=1.0, durs=None, pitch=None):
    with torch.no_grad():
        tokens = fastpitch.parse(inp)
        spec, _, durs_pred, _, pitch_pred, *_ = fastpitch(text=tokens, durs=durs, pitch=pitch, speaker=None, pace=pace)
        audio = hifigan.convert_spectrogram_to_audio(spec=spec).to('cpu').numpy()
    return spec, audio, durs_pred, pitch_pred
```

I want to change the parameter speaker to 0 or 1, but I get the following error:

```
[NeMo E 2022-03-09 19:12:15 vocabs:324] Torch distributed needs to be initialized before you initialized <nemo.collections.asr.data.vocabs.Phonemes object at 0x7f876a7febd0>. This class is prone to data access race conditions. Now downloading corpora from global rank 0. If other ranks pass this before rank 0, errors might result.
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
[<ipython-input-69-2fd102bd26e0>](https://localhost:8080/#) in <module>()
      5   phraseNumber = i+1
      6   with torch.no_grad():
----> 7       spec_norm, audio_norm, durs_norm_pred, pitch_norm_pred = str_to_audio(p)
      8 
      9       # Let's try to make the speech more solemn

9 frames
[/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py](https://localhost:8080/#) in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
   2042         # remove once script supports set_grad_enabled
   2043         _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-> 2044     return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
   2045 
   2046 

TypeError: embedding(): argument 'indices' (position 2) must be Tensor, not int
```

Thank you in advance.