Error loading Llama model
I got a problem loading the Llama-2-13b-hf model with the following code,LlamaForCausalLM.from_pretrained(base_model, trust_remote_code=True, device_map = “cuda:0”, load_in_8bit = True),A error returned asImportError: Usingload_in_8bit=Truerequires Accelerate:pip install accelerateand the latest version of bitsandbytespip install -i https://test.pypi.org/simple/ bitsandbytesor pip install bitsandbytes`.But I am pretty sure the two package are installed with version bitsandbytes-0.41.3, accelerate 0.25.0. Anyone has encountered the issue before and know how to resolve it, thank you very much!
2023-12-27T18:38:52Z
Hi,Are you running in Google Colab? If yes, restarting the runtime may help.
2023-12-27T19:21:14Z
Hi, nielsr, I am running it in AWS sagemaker, I found a solution in forum by downgrading transformers to 4.30.0, the error disappears but a new error emerge: SafetensorError: Error while deserializing header: HeaderTooLarge, wonder if you have any clue on that, thank you!
2023-12-27T19:35:18Z
It should also work with the latest Transformers version. Could you try creating a new environment?
2023-12-27T20:14:31Z
Yes, latest version of transformers also pass that error, thank you very much!
2024-03-09T08:14:43Z
Hi, were you able to resolve the error?