How to use specified GPUs with Accelerator to train the model?
I’m training my own prompt-tuning model using transformers package. I’m following the training framework inthe official exampleto train the model. I’m training environment is the one-machine-multiple-gpu setup. My current machine has 8 gpu cards and I only want to use some of them. However, theAcceleratorfails to work properly. It just puts everything on gpu:0, so I cannot use mutliple gpus. Also,os.environ['CUDA_VISIBLE_DEVICES']fails to work.I have re-written the code without usingAccelerator. Instead, I usenn.Dataparallelwithos.environ['CUDA_VISIBLE_DEVICES']to specify the gpus. Everything work fine in this case.So what’s the reason? According the manual, I thinkAcceleratorshould be able to take care of all these things. Thank you so much for your help!FYI, here is the version information:python 3.6.8transformers 3.4.0accelerate 0.5.1NVIDIA gpu cluster
2021-10-21T11:56:39Z
Acceleratordoes not useDataParallelon purpose since it’s not recommended by PyTorch. Have you properly set up your config inaccelerate configand launched your script withaccelerate launch?Alternatively, did you launch you script withpython -m torch.distributed.launch ...? See more commandshere.
2021-10-21T14:11:15Z
Thanks for you reply! I tried to useaccelerate config, but I haven’t found a place to specify the gpu cards that I want to use. For example, if I setnproc_per_nodeto 4, it will automatically use gpu:0, gpu:1, gpu:2, gpu:3 on my machine. Is there a way to change this behavior?Thank you so much~
2021-10-21T14:16:59Z
No, you will also need to add CUDA_VISIBLE_DEVICES=“0, 1, 2, 3” when launching, to use those four GPUS.
2021-10-21T15:31:18Z
Yes, I actually done this by settingos.environ['CUDA_VISIBLE_DEVICES'] = "3,4,5,6"at the beginning of my code. But it doesn’t work. Did I miss anything?Thank you for your help!
2021-10-21T15:32:43Z
No it needs to be done before the lauching command:CUDA_VISIBLE_DEVICES = "3,4,5,6" accelerate launch training_script.py
2021-10-21T15:41:15Z
image1689×903 47.9 KBStill fails to work correctly
2021-10-21T15:44:03Z
Why do you say that? It seems good to me.
2021-10-21T15:48:20Z
Oach, sorry. I just check the gpu state. It’s great. I just stupidly thought theDeviceshould showcuda:3/4/5/6(it shouldn’t of course since only 4 gpus are visible).Thank you so much for your quick reply. Your help really save me since it’s my first time to useacceleratepackage.
2021-10-21T15:52:31Z
Yes, you can’t trust completely the devices printed
2022-06-29T11:58:03Z
Sir, I have this error. Can you please suggest me the solution of this errorGPUAccelerator can not run on your system since the accelerator is not available. The following accelerator(s) is available and can be passed intoacceleratorargument ofTrainer: [‘cpu’].
2022-10-16T13:29:58Z
Hi,I also have the same error as yours. Have you found the solution? Hope for your reply!
2022-10-31T06:34:58Z
Hi,What the second floor says works for me. A workable command looks like thisCUDA_VISIBLE_DEVICES={gpus you gonna use} python -m torch.distributed.launch --nproc_per_node={the number of gpu used} \
  your_python_script.py {other arguments for your python script}Sorry for the late reply, hope you have solved the problem.
2022-10-31T09:50:38Z
accelerate launchalso now lets you specify--gpu_idsas a string too
2023-04-10T12:43:46Z
Why don’t you just make our life easier and simply add a parameter to the trainer to get the GPU id or list of ids?
2024-08-23T11:07:22Z
This does not work on windows prompt