Customize tokenizer in model card's widget
I trained a Chinese Roberta model. In the model card, the widget uses a tokenizer defined in config.json(RobertaTokenizer). But my model usesBertTokenizer. Can I customize the tokenizer in the widget of the model card just like I can choose any combination of model and tokenizer in a pipeline?I tried to useBertModelinstead ofRobertaModel(copy weights from Roberta to Bert). But the position embedding is different. And the outputs are different… So I have to use this combination ofRobertaModelandBertTokenizer. Is that mean I can’t use the inference widget?
2020-11-03T08:30:15Z
We are working on adding this indeed (as I understand it’s the same ashttps://github.com/huggingface/transformers/issues/8125)