Does one need to load the model to GPU before calling train when using accelerate?
Do I need to call:device = torch.device(f"cuda:{0}" if torch.cuda.is_available() else "cpu")
        model = model.to(device)if I am using accelerate? e.g.,accelerate launch --config_file REDACTED_TOKEN.pyI was worried about theaccelerator.prepare(...)not working as expected.related question:huggingface transformers - How to adapt LLaMA v2 model to less than 7B parameters? - Stack Overflowcross:huggingface transformers - Does one need to load the model to GPU before calling train when using accelerate? - Stack Overflow
2023-09-29T19:42:25Z
HF always loads your model to GPU if it’s available. With accelerate it does it too.The Trainer checks torch.cuda.is_available() to detect if GPUs are available.refhttps://github.com/huggingface/transformers/blob/bffac926ca6bc6c965a92bfbfd00c567a2c0fb90/src/transformers/trainer.py#L2298so# from accelerate import Accelerator
        # accelerator = Accelerator()
        # # self.is_deepspeed_enabled = getattr(accelerator.state, "deepspeed_plugin", None) is not None
        # is_fsdp_enabled = getattr(accelerator.state, "fsdp_plugin", None) is not None
        # if not is_fsdp_enabled: # not sure if this is needed but its for sure safer
        #     # maybe figuring out how to run everything with accelerate would fix things...
        #     # ref: https://stackoverflow.com/questions/77204403/does-one-need-to-load-the-model-to-gpu-before-calling-train-when-using-accelerat
        #     device = torch.device(f"cuda:{0}" if torch.cuda.is_available() else "cpu")
        #     model = model.to(device)is not needed. Just load the model and trainer will take care of it.
2023-10-31T20:38:35Z
related HF discord:Discord