Tutorial on Pretraining BERT
I would like to pretrain a BERT model with custom data. For example, data  in my native language. Is  there any example script about pretraining a BERT model? I found the tutorial with the EsperBERTo model but this is not a BERT model as far as I understand.
2020-12-15T09:01:11Z
You can have a look at the example scripts here:https://github.com/huggingface/transformers/tree/master/examples/language-modeling#robertabertdistilbert-and-masked-language-modeling