Tokenizer for 'sshleifer/distilbart-xsum-12-6'?
I try to generate outputs usingsshleifer/distilbart-xsum-12-6but it gives me the following error:OSError: Model name 'sshleifer/distilbart-xsum-12-6' was not found in tokenizers model name list (facebook/bart-base, facebook/bart-large, facebook/bart-large-mnli, facebook/bart-large-cnn, facebook/bart-large-xsum, yjernite/bart_eli5). We assumed 'sshleifer/distilbart-xsum-12-6' was a path or url to a directory containing vocabulary files named ['vocab.json', 'merges.txt'], but couldn't find such vocabulary files at this path or url.I am thinking instead oftokenizer = AutoTokenizer.from_pretrained("sshleifer/distilbart-xsum-12-6")we should change it to another tokenizer for xsum? Thank you!(Again, thank you Sam for your wonderful work.
2020-07-30T04:49:27Z
chz816:tokenizer = AutoTokenizer.from_pretrained(“sshleifer/distilbart-xsum-12-6”)Hi@chz816, this call should work. You can also useBartTokenizer
2020-08-18T20:41:13Z
I can download the tokenizer with your code. strange.