How to use hugging face to fine-tune ollama's local model
I am a newbie. I have downloaded ollama and can run gemma:2b on my laptop.model, I want to fine-tune this model, but I did not find the gguf file under C:/Users/lihuacat/.ollama/models. I found one under C:/Users/lihuacat/.ollama/models/manifests/registry. .ollama.ai/library/gemma/2b, but when writing the file, an error that the file cannot be found will be reported.C:/Users/lihuacat/.ollama/models/blobs/sha256-c1864a5eb19305c40519da12cc543519e48a0697ecd30e15d5ac228644957d12 This file, but when writing the file name, an error that the config file cannot be found appears. Is it true that ollama can only be used as a proxy? The model file needs to be downloaded from hugging face. Can?
2024-05-10T14:00:54Z
OSError: Unable to load weights from pytorch checkpoint file for ‘C:/Users/lihuacat/.ollama/models/manifests/registry.ollama.ai/library/gemma/2b’ at ‘C:/Users/lihuacat/.ollama/models/manifests/registry.ollama.ai/library/gemma/2b’. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.
2024-05-10T14:01:31Z
image1920×1080 284 KB
2024-05-10T14:29:09Z
Hi,File formats like GGUF are typically meant for inference on local hardware, seeggml/docs/gguf.md at master · ggerganov/ggml · GitHub.For fine-tuning models, one typically uses one of the following libraries (in combination with GPU hardware):Transformers, TRL, PEFT. These are libraries developed by HF making it very easy to fine-tune open-source models on your custom data.Unsloth:GitHub - unslothai/unsloth: Finetune Llama 3.1, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory. Claims to fine-tune models faster than the Transformers library.Axolotl:GitHub - axolotl-ai-cloud/axolotl: Go ahead and axolotl questions. It works with a YAML-based configuration file that defines your entire fine-tuning. See e.g.axolotl/examples/llama-3 at main · axolotl-ai-cloud/axolotl · GitHubfor llama-3Torchtune:https://github.com/pytorch/torchtune:a new library developed by Meta for LLM fine-tuning.After fine-tuning, the weights can be converted to the GGUF format, which allows local inference with ollama and llama cpp. SeeHow to convert any HuggingFace Model to gguf file format? - GeeksforGeeksfor a tutorial.Additionally there’s Apple’sMLXlibrary which allows to fine-tune LLMs on a Macbook. Seemlx-examples/llms at main · ml-explore/mlx-examples · GitHubfor fine-tuning LLMs.
2024-05-11T01:58:04Z
thank you，now i know how to over my job
2024-07-01T08:39:24Z
所以这个具体什么情况导致的呢?我在vscode调用llama3的时候也出现相关问题,是需要再去hugging face上把分词器文件那些下载了才行吗
2024-08-02T13:15:32Z
@lihuacatDid you find any way to fine tune the model which is avail in ollama?
2024-08-28T03:37:49Z
Deepadharshini:Did you find any way to fine tune the model which is avail in ollama?I am facing lot of issues to fine tune ollama models but, i tried with HF models after that i load those models into ollama but i found nonsense anwers.