"Can't load tokenizer for <model>" after download
I’m trying to use thecardiffnlp/twitter-roberta-base-hatemodel on some data, and was following the example on themodel’s page. The script works the first time, when it’s downloading the model and running it straight away, but after I run it after this I get the message:Can't load tokenizer for 'cardiffnlp/twitter-roberta-base-hate'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'cardiffnlp/twitter-roberta-base-hate' is the correct path to a directory containing all relevant files for a RobertaTokenizerFast tokenizer.It’s downloaded the model into a subdirectory of my working directory, which it’s presumably finding. That directory contains two files:config.jsonandpytorch_model.bin. If I delete that directory it works again for one run, but as it’s a half-gig model I’d rather not have to do that each time!
2023-10-21T07:46:08Z
Hello, have you solved this problem?
2023-10-24T11:18:58Z
Hi, I’m sorry but I think I just used a different model. In this case the Cardiff Hatelab had a more up to date model anyway which was probably better maintained.Good luck
2023-12-13T14:14:13Z
Hey,The community faced the same issue atVenmo Nerdbut I tried solving the issue via help of my friends.