Gated Model access error during training on SageMaker
I am trying to run a training job with my own data on SageMaker using HugginFace estimator.I have accepted T&C on the model page,I do a hugging face loginfrom huggingface_hub import notebook_login
notebook_login()Run fit methodgit_config = {'repo': 'https://github.com/huggingface/transformers.git','branch': 'v4.26.0'}huggingface_estimator = HuggingFace(
entry_point='run_clm.py',
source_dir='./examples/pytorch/language-modeling',
instance_type='ml.p3.2xlarge',
instance_count=1,
role=role,
git_config=git_config,
transformers_version='4.26.0',
pytorch_version='1.13.1',
py_version='py39',
hyperparameters = hyperparameters
)

huggingface_estimator.fit()But i get access token error.HTTPError: 401 Client Error: Unauthorized for url:Cannot access gated URL repo for URLhttps://huggingface.co/bigcode/starcoder/resolve/main/config.json.OSError: bigcode/starcoder is not a local folder and is not a valid model identifier listed on ‘Models - Hugging Face’ . If this is a private repository, make sure to pass a token having permission to this repo withuse_auth_tokenor log in withhuggingface-cli loginand passuse_auth_token=True.I have tried to pass the auth token in HuggingFace estimator but has not worked, tried to pass it as hyperparameter as a hub_token but that didnt work.‘hub_token’: HfFolder.get_token()How can i successfully run the fit method?
2023-09-04T05:33:51Z
Same problem with LLAMA model even I have a PRO subscription.