I encountered the error below mentioned while attempting to merge the adapter with the base model. I have attached my code for fine-tuning the model. Any assistance in resolving the issue would be greatly appreciated
notebooklink.RuntimeError: unable to mmap 9976576152 bytes from file </home/devuser/.cache/huggingface/hub/models–meta-llama–Llama-2-7b-chat-hf/snapshots/c1b0db933684edbfe29a06fa47eb19cc48025e93/model-00001-of-00002.safetensors>: Cannot allocate memory (12)
2024-02-21T04:23:59Z
Were you able to solve it? I am facing the same issue
2024-02-21T13:18:29Z
Yep, I performed the merge adapter part in a separate file and I loaded the model with low_cpu usage.
2024-02-22T06:55:10Z
Thanks!But wouldn’t this offload the entire model on the CPU, even if you have a GPU?
2024-02-22T11:16:10Z
I don’t think so. Model offload scenarios can be handled by using the ‘device_map’ argument.