How to use llm (access fail)
meta-llama/Meta-Llama-3.1-8B-InstructI follow the tutorial and it show the error.‘’’Cannot access gated repo for urlhttps://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct/resolve/main/config.json.Access to model meta-llama/Meta-Llama-3.1-8B-Instruct is restricted. You must be authenticated to access it.‘’’Q2.I download the Meta-Llama-3.1-8B-Instruct by git cloneHow to load it with the code?I dont see any solution?pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)
2024-08-20T08:18:23Z
hi@alice86You need to agree the license agreement to access this model. You can find it in the model card.FAQhuggingface.cometa-llama/Meta-Llama-3.1-8B-Instruct · Access request FAQYou can get the Meta Llama models directly from Meta or through Hugging Face or Kaggle. However you get the models, you will first need to accept the license agreements for the models you want.You will need this one:huggingface.cometa-llama/Meta-Llama-3-8B-Instruct · Denied permission to DLHF indicates "Gated model You have been granted access to this model"
2024-08-21T00:15:44Z
Q1I already accept the license.And the official don’t show any method to download it.It just direct usepipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)Q2It is different question.If I download the model and how to load  model  without token? (already download model)Because I already download the model.I expect the result ispython test.py    => it will automatically download model with tokenpython test.py  -m ./model/llama3 => It will use the model that I already download without any token.In fact, I dont see any code for   python test.py  -m ./model/llama3
2024-08-21T11:10:54Z
hi@alice86About your first question:Did you create an access token? Did you add a permission for relevant repository?You can check fromHugging Face – The AI community building the future..(Edit permissions → Repositories permissions)You need to run something like this:pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
    token="REDACTED_TOKEN",
)huggingface.coUser access tokensWe’re on a journey to advance and democratize artificial intelligence through open source and open science.
2024-08-21T11:31:20Z
About your second question:You need to call the pipeline withmodel='./path_for_local_model':pipeline = transformers.pipeline(
    "text-generation",
    model='./path_for_local_model',
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)