transcribe() outputting different results than validate()
Hello,

I have fine-tuned the German Conformer Transducer on a subset of Mozilla Common Voice. The WER on the validation set decreases throughout the 20 training epochs. However, when I load the checkpoint and use the `transcribe` method, the model outputs weird predictions containing `??`. For example: 

```
' ⁇ un müssen meines  ⁇ rachtens konkrete und erkennbare  ⁇ aßnahmen ergriffen werden ⁇ '
```

This does not occur when I use the pre-trained model offered by Nvidia. Only when I use the one I have fine-tuned myself. I have run another validation loop with the same checkpoint, and the WER is lower than for the model that is not fine-tuned. 

I assume that there must be a difference between these two methods (`validate` and `transcribe`) that makes one work but not the other. Do you have any clue as to why this is?