Importing DataCollator gives me a Bus error
Simply running this linefrom transformers import DataCollatorForLanguageModelinggives me aBus error (core dumped).I’m usingtransformers==4.23.1andpython==3.9.12. What could be the cause of this error?
2022-11-03T18:16:54Z
My/dev/shmwas full apparently. After increasing the size of it by runningmount -o remount,size=25G /dev/shm, the import worked just fine. I’ve no idea why this particular import statement usesshmthough
2023-11-24T12:04:19Z
How much should be the minimum /dev/shm. I have 8GB and it is empty. I am still getting Bus Error while importing sentence_transformers.