Wrong inference for repeated word in RNNTs
**Describe the bug**

I'm building an ASR with Conformer Transducer with "beam" decoding strategy beam_size of 2 but when I inference some audio contains a repeat number (telephone, ID..) the ASR model is often missing the number.
For example: 
one one one two two three three  => one one two three
nine nine nine nine  => nine nine
...

I thought it was because of the decoding strategy so I tried to change decoding strategy to "greedy" but so many repeated words are generated (eg: one one two two => one one oneo one one two two two)

I also try ASR with Adapters by training only audio with repeated number only but I doesn't solve problem well

**Expected behavior**

ASR model should generate enough number even the number or words are repeated

**Environment details**
-OS version - Ubuntu 18.04.5 LTS
-PyTorch version - 1.10.1
-Python version - 3.8.11

**Others**
-Decoding strategy: beam decoding with beam_size: 2

How can I solve this problem?
Thanks!

