RNN-T decoding: question about greedy batch inference
Yes, this is correct. I didn't think about reusing blank_mask for the label decoding step. Would you like to submit a PR to improve this for both greedy decoding implementations?
As `blank_mask` contains ones where `time_idx >= out_len` and `k == self._blank_index` and zeros otherwise, we can replace the following code:

https://github.com/NVIDIA/NeMo/blob/5b6dd3850129898be47cf0d65587897ec45a5b59/nemo/collections/asr/parts/rnnt_greedy_decoding.py#L451-L454

with the following code (without masked_fill):

```python
for kidx, ki in enumerate(k):
    if blank_mask[kidx] == 0:
        label[kidx].append(ki)
```

It allows to remove `time_mask` at all. The following block:

https://github.com/NVIDIA/NeMo/blob/5b6dd3850129898be47cf0d65587897ec45a5b59/nemo/collections/asr/parts/rnnt_greedy_decoding.py#L391-L392

Can be squeezed to:

```python
blank_mask = time_idx >= out_len
```

Is it correct? @titu1994 

