CUDA out of memory for Longformer
I have issue training the longformer on custom dataset, even on a small batch number, it says CUDA out of memory,RuntimeError                              Traceback (most recent call last)in ()----> 1 trainer.train()18 frames/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in _pad(input, pad, mode, value)3550     assert len(pad) // 2 <= input.dim(), ‘Padding length too large’3551     if mode == ‘constant’:-> 3552         return _VF.constant_pad_nd(input, pad, value)3553     else:3554         assert value == 0, ‘Padding mode “{}”" doesn’t take in value argument’.format(mode)RuntimeError: CUDA out of memory. Tried to allocate 1.13 GiB (GPU 0; 15.90 GiB total capacity; 11.40 GiB already allocated; 659.81 MiB free; 14.39 GiB reserved in total by PyTorch)
2021-10-20T13:32:47Z
Did you try smaller batch sizes? What is the size of single batch size in your RAM?
2021-10-20T13:44:37Z
I tried 8 batch size, do not remember the single batch size. I use colab so it has its own limitations even the Pro version. any thoughts?
2021-10-20T14:50:19Z
Your input sentences are being limited to a maximum size?
2021-10-22T11:26:09Z
I truncated to lesser as well.
2021-10-22T12:00:07Z
Could you provide a simple snippet to reproduce the OOM?
2021-10-22T13:06:21Z
if you see my post above, its months ago, its not actively something I am working on, I didn’t get answer but I solved that time by having smaller batches and less input.I have few more issues like batch processing, having predictions at the end with a score, do you mind discussing that?