CUDA is out of memory
HiI finetune xml-roberta-large according tothis tutorial. I met a problem that during training colab CUDA is out of memory.RuntimeError: CUDA out of memory. Tried to allocate 978.00 MiB (GPU 0; 14.76 GiB total capacity; 12.62 GiB already allocated; 919.75 MiB free; 12.83 GiB reserved in total by PyTorchAnd it is given that batch_size = 1I tried to do that on xml-roberta-base, training lasts longer but over all ends up with the same problem. I tried bert-base-uncase that is in tutorial and it’s okay. But my data is multillingual!I want to understand is it true that this problem is just because of natural limits of Colab or it is my fault. Is it possible to finetune xml roberta large in Colab?Thanks!
2021-03-11T20:01:55Z
Hi@Constantin, it’s possible that you’re getting allocated one of the K80 GPUs on Colab which probably doesn’t have enough RAM to handlexlm-roberta-large.You can “cheat” you way to a better GPU (either Tesla T4 or P100) by selectingRuntime > Factory reset runtimein the settings:Screen Shot 2021-03-11 at 8.58.12 pm2150×1364 316 KBYou can check what kind of GPU your notebook is running by executing the following in a code cell:!nvidia-smi
2021-03-12T08:57:06Z
You could also try kaggle. It’s very similar and gives you a P100 (I think they give you more memory as well).
2023-10-09T06:12:49Z
Tried using kaggle’s P100, but I am getting the same error, i.e.OutOfMemoryError: CUDA out of memory. Tried to allocate 384.00 MiB (GPU 0; 15.90 GiB total capacity; 14.70 GiB already allocated; 245.75 MiB free; 14.78 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONFAny advice on using another model for sequence classification task rather than xlm?