Finetuning ASR on another language (tokenizer)
Hello, 

I would like to finetune the stt_en_conformer_transducer_medium on 200 hours of another language (romanian). I am at the tokenization step and I am unsure of what would be the best practice:

1. Prepare the tokens using the text from the training audio transcripts (around 2 million words, not necessarily different). I would use sentencepiece, with unigram and vocab of 1024.
2. Prepare the tokens using a big text corpus of around 50 million words, one that is usually used for training LMs. Same configuration.

I am unsure if it is ok to train the tokenizer on more text than the training audios, I was not able to find if this is an OK practice and can lead to better/worse results. I was thinking that it might generalize better if it has more possible tokens. The tokenizer I obtained on just 200 hours looked rather limited, with lots of big words that are not actually that common outside of the training set. However, I am scared that it might not learn to pronounce some tokens based on the audio dataset, but I consider this rather unlikely.

One last question would be: What are the "##" , "_", "_de", etc. tokens representing? 

Thank you in advance!