What's the difference between bart-base tokenizer and bart-large tokenizer
Hi,tokenizer1 = BartTokenizer.from_pretrained('facebook/bart-base')
tokenizer2 = BartTokenizer.from_pretrained('facebook/bart-large')What’s the difference conceptually? I can understand the diff in uncased and cased ones for bert.But why this?btw, bart base and large have the same “vocab_size”: 50265 in their config.Thanks.
2020-12-05T16:11:18Z
It is obviously related to more number of parameters used in the bart-large as mentioned in the description.facebook/bart-large24-layer,1024-hidden, 16-heads, 406M parametersfacebook/bart-base12-layer,768-hidden, 16-heads, 139M parameters
2020-12-05T16:54:41Z
Thanks for reply. but why is a tokenizer dependent on the number of model’s parameters?  isn’t it just responsible for text tokenization for corpus and not related to model’s size?
2020-12-05T17:01:58Z
Easy there with the “obviously”. This isn’t obvious, because as@zuujhytrightfully says, the number of parameters is typically not directly related with the vocab. I.e. the vocab embeddings index often do not change between small/large models, but the model’s blocks get wider and/or deeper. I think this is a good question.cc@patrickvonplaten
2020-12-05T17:13:15Z
Agree, would like to know more about it
2020-12-06T17:14:03Z
Those tokenizers are identical. You can check it by just comparing the files over athttps://huggingface.co/facebook/bart-base/tree/mainandhttps://huggingface.co/facebook/bart-large/tree/mainIncidentally, they’re also the same as the ones forroberta-*models.We duplicate tokenizers into their models for ease of use (a model id is all you need)
2020-12-06T18:28:23Z
I understand, thanks!