Running LLaMA 3.1 8B Model Downloaded from Meta - Missing Configuration File
Hello everyone,I recently downloaded the LLaMA 3.1 8B model directly from Meta’s website using their provided link. I’m trying to implement this model on Kaggle/Colab for testing purposes before investing in dedicated hardware.Current SetupI have downloaded the following files:consolidated.00.pthparams.jsonchecklist.chktokenizer.modelWhen trying to load the model locally (pointing to the files’ location), I’m getting an error indicating thatconfig.jsonis missing. I’m specifically trying to run this directly on Kaggle without using tools like Ollama or other third-party software.What additional files or configurations do I need to get this working?Is there a specific way to point to these files when implementing them on Kaggle?Are there any additional steps I need to take to properly set up these model files?I would greatly appreciate guidance on how to proceed with implementing these files directly from Meta’s website into a Kaggle environment.Thank you in advance for your help!
2024-12-30T02:17:06Z
The Hugging Face format is a model per folder. It is better to avoid handling it on a file-by-file basis.Download the entire directoryand use it. Alternatively, you can download itby loading it directly from HF using from_pretrained()or similar.
2024-12-30T02:33:49Z
Thank you for your response. I would like to clarify a few points:My current files were downloaded directly from Meta’s official website (not from Hugging Face). These are the files I received:- consolidated.00.pth
- params.json
- checklist.chk
- tokenizer.modelRegarding loading from HF using from_pretrained():Could you provide an example of the correct code to load the LLaMA 3.1 8B model on Kaggle? Would it be something like:from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "meta-llama/Llama-3.1-8b-hf"  # Is this the correct path?

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)Most importantly, how can I download and save the model locally so I don’t need to re-download it every time I restart Colab? Is there a way to:Download the complete model folder structure from HFSave it locallyLoad it from local storage in future sessionsI’m new to working with LLMs and would appreciate guidance on the proper setup process, especially for maintaining a local copy of the model.Thank you!
2024-12-30T02:42:51Z
2Like this.REDACTED_TOKEN = "hf_******" # You can bring it from anywhere...
#model_name = "meta-llama/Llama-3.1-8b-hf" 
model_name = "meta-llama/Llama-3.1-8B"  # from https://huggingface.co/meta-llama/Llama-3.1-8B
tokenizer = AutoTokenizer.from_pretrained(model_name, token=REDACTED_TOKEN) # Llama3 is a gated model, so you need a token.
model = AutoModelForCausalLM.from_pretrained(model_name, token=REDACTED_TOKEN) # same above3There are several methods, but I recommend snapshot_download because it’s the easiest and most efficient. There is alsoa method that uses git.huggingface.coDownloading filesWe’re on a journey to advance and democratize artificial intelligence through open source and open science.About loding from local:model_path = "./llama3-8b" # your downloaded model directory path
model = AutoModelForCausalLM.from_pretrained(model_path)