LM Example run_mlm.py. The memory usage of each GPU is inconsistent
When I use run_mlm.py.The memory usage of GPU number 0 is twice that of other GPUs.Is this normal?Thanks for any advice!
2023-05-18T11:54:45Z
May be I used fp16?
2023-09-06T16:23:08Z
I would also like to know what to do about this.