Memory leak in 01_ASR_with_NeMo.ipynb
Thanks for finding this, I'll send a PR tomorrow patching this. If you only need inference, you can freeze the model with model.freeze() and this will prevent any leak. 

It's the notebook environment which has the potential for leaking cuda tensors, and scripts don't have this leakage.

The notebooks are meant to be tutorials, not full scale training scripts. For such scripts please visit the examples/asr/ directory. You might need to modify the scripts to suit finetuning purposes. 

The model uses batch norm, and is therefore affected by padding inside samples of uneven length. For academic work, we use batch size 1 inference on a single gpu. 

We do not support TPU training and it is not currently tested.  Thanks! I really appreciate the quick response. To clarify, I got the prints before and after to match by adding:

del test_batch, log_probs, targets, targets_lengths, encoded_len, greedy_predictions

But maybe not all those deletes are needed? I hope that helps!
 While I have you, I saw in some other comments that "iter_with_steps" can help with cuda memory issues. Where would I change that param? Thanks again! That argument has been deprecated for quite some time, and it has no impact on memory. If you need to limit training, you can use the PTL trainer argument "max_steps"
Running 01_ASR_with_NeMo.ipynb WER cells appears to be leaking data in CUDA. With batch_size 32, using print(torch.cuda.memory_allocated()) prints 91,033,600 prior to running, then 129,805,824 after. There's a hard delete in https://github.com/NVIDIA/NeMo/blob/8e239ecc7d7bc1206c72daa207b0f122e85551f1/examples/asr/speech_to_text_infer.py that's missing in the notebook. 

Additionally, a few more questions: 

1) Is there some other way to run WER? 

2) Why does batch size change the WER? Isn't it averaged across batches?

3) Is there any way to run this without Cuda, e.g., so that it can be used on Colab TPU's

Thanks! I have really appreciated getting to use this!
