Cannot export Fast Pitch models
**Describe the bug**

I try to export fast pitch models, but it took a lot of RAM then process is killed. In my case, I use Mac OS to export and swap increase from 3 GB to 54 GB. I also used to try with google colab, but the kernel died after run export because memory issue.

**Steps/Code to reproduce bug**

```
from nemo.collections.tts.models import FastPitchModel

fastpitch = FastPitchModel.from_pretrained("tts_en_fastpitch")
fastpitch.export("fastpitch.onnx")
```

**Expected behavior**

The models can be exported to onnx

**Environment overview (please complete the following information)**

 - Environment location: Intel(R) Core(TM) i5-8279U CPU @ 2.40GHz
 - Method of NeMo install: pip install nemo_toolkit==1.6.2

**Environment details**

- OS version: MacOS 12.0.1
- PyTorch version: 1.10.1
- Python version: 3.8.9


Thank you