Value error : sentencepiece
Hi,I have value error: This tokenizer cannot be instantiated. Please make sure you havesentencepieceinstalled in order to use this tokenizer.I installed and updatedsentencepiece(0.1.95) but still getting the same error. Can someone help?Thank you!
2021-03-11T20:27:21Z
Hi@Katarina, what happens if you try installingtransformersin a new environment withpip install transformers[sentencepiece]Does that solve the problem?
2021-03-12T07:58:48Z
Hi, thanks on answer…I tried with this but still the same error
2021-03-12T08:23:20Z
Can you please share the code you are running and the full stack trace / error message?
2021-03-12T11:11:27Z
Acctually it is working after I restarted my enviroment…thank you!
2022-02-10T01:57:38Z
This works:pip install sentencepiece
2022-11-06T17:07:40Z
use this one-pip install transformers[sentencepiece] datasets
2023-02-13T16:31:32Z
this has to do with you shell actually. You are probably using zsh if you are on Mac, and you are gettingzsh: no matches found: transformers[sentencepiece].Solution:pip install "transformers[sentencepiece]"
2023-06-23T05:44:09Z
Fix:pip install transformers[sentencepiece]if it is installed and still getting error, restart the kernel
2023-11-16T13:54:42Z
Kernel Restart solved this problem.
2023-12-02T04:03:05Z
Thank you!
2024-01-29T05:13:12Z
Use pip install transformers[sentencepiece]And restart the kernel. It will work
2024-08-10T22:33:07Z
I also found that using conda install works. I had to do conda install sentencepiece and conda install protobuf to get it to work for me.
2024-09-24T05:48:43Z
Besides conda install sentencepiece, I found that I can do conda install sentence-transformers which pull in sentencepiece also.
2024-10-25T17:50:34Z
Yes, I met this issue either.My platform is below:MacOS 15.0.1Python 3.12.6pip 24.2Then I executed this command:pip3 install sentencepieceAnd restarted Kernel, then it works