Can't set attribute 'device', for some reason i need to train model on only one gpu on a mutlti gpu machine
'm doing some lora experiments on my 8-GPU machine, while im running other training process on 6 of 8 gpus. And I cant use CUDA_VISIBLE_DEVICES for i need to launch the trainging by using multiFile “/home/asasa/repo/kohya_ss/train_network.py”, line 222, in trainaccelerator.device = torch.device(“cuda:7” if torch.cuda.is_available() else “cpu”)File “/home/asasa/repo/kohya_ss/train_new.py”, line 302, in train_loratrainer.train(args)File “/home/asasa/repo/kohya_ss/train_new.py”, line 733, in total_traintrain_args = train_lora(train_dataset_dir, name, lora_dir, log_dir)AttributeError: can’t set attribute ‘device’threads. If I set the env var, that would work like a two gpus distributed training. But i need train different exp on different gpu.So I decide to set thedeviceattribute straightly when creating the Acclerator object, but i failed for there is no params named device or sth. And then I tried set the attribute after created the Acclerator object, it fails too, like theerrorabove.My train process need one gpu for each, so i have to make them seperate. is there anyway to set the device before the train process?
2024-09-16T16:25:14Z
Hi@Yuhyeong. Were you able to figure this out ?
2024-09-16T21:23:58Z
I have no experience in actual use since I only have one crappy GPU, but it seems to be the usual practice to specify device_map if you want to use multiple GPUs at will. For more information, please search “device_map”.stackoverflow.comChoose available GPU devices with device_mapmachine-learning, transformer-modelasked byssamtkwonon10:04AM - 09 Aug 24 UTCgithub.com/huggingface/transformersHow to Use Transformers pipeline with multiple GPUsopened07:08PM - 23 Feb 22 UTCclosed03:06PM - 07 Apr 22 UTCvikramtharakanI'm using an tweaked version of the `uer/roberta-base-chinese-extractive-qa` mod…el. While I know how to train using multiple GPUs, it is not clear how to use multiple GPUs when coming to this stage. Essentially this is what I have:

```
from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline
os.environ["CUDA_VISIBLE_DEVICES"] = "0"

model = AutoModelForQuestionAnswering.from_pretrained('uer/roberta-base-chinese-extractive-qa')
tokenizer = AutoTokenizer.from_pretrained('uer/roberta-base-chinese-extractive-qa')
QA = pipeline('question-answering', model=model, tokenizer=tokenizer, device=0)
```

Except the model I'm using is a tweaked version of the base model shown above. I'm trying to process a large number of documents and at the rate it's going at it would take on the order of weeks. Is there anyway to set the pipeline to use multiple gpus? So I could use something like

```
os.environ["CUDA_VISIBLE_DEVICES"] = "0,1"
device = torch.device("cuda")
```
And then use both devices?

Also looks like someone else had the same question: https://stackoverflow.com/questions/64193049/how-to-use-transformers-pipeline-with-multi-gpu