AssertionError: 80, 1: /home/user/Documents/LJSpeech-1.1/wavs/LJ012-0230.wav
I've been trying to train fastpitch + hifigan e2e with `fastpitch_hifigan_e2e.py`

I have used the script at https://github.com/NVIDIA/DeepLearningExamples/blob/master/PyTorch/SpeechSynthesis/FastPitch/prepare_dataset.py to create the durations and pitch .pt files. On starting the training, I get the assertion error. 

```sh
File "/home/user/Labs/NeMo/nemo/collections/asr/data/audio_to_text.py", line 758, in _collate_fn
    assert len(durs_i) == len(pitch_i), f"{len(durs_i)}, {len(pitch_i)}: {path_i}"
AssertionError: 80, 1: /home/user/Documents/LJSpeech-1.1/wavs/LJ012-0230.wav
```
It looks like there is a mismatch in the lengths of duration and pitch. It would be great if I could get some help with this. 
