Lama 3.23b performs great when I download and use using ollama but when I manually download the model or if I use the gguf model by unsloth, it gives me irrelevant response. Please help me out
Hi, Im very new to this but quite interested about LLM’s, I’m working on a project that requires me to fine tune an LLM but the issue is the gguf models of Llama 3.23b that I download and try to run give me weird outputs, like the one below.image1056×492 124 KBBut when I use the one from ollama itself(command: ollama run llama3.2)… it works fine
2024-10-30T23:35:48Z
It seems that Ollama internally loads theInstructmodel instead ofBase.How to Configure LLaMA-3:8B on HuggingFace to Generate Responses Similar to Ollama?🤗TransformersHi HuggingFace Community, 
I have been experimenting with the LLaMA-3:8B model using the following code: 
import transformers
import torch

model_id = "meta-llama/Meta-Llama-3-8B"

pipeline = transformers.pipeline(
    "text-generation", model=model_id, model_kwargs={"torch_dtype": torch.bfloat16}, device_map="auto"
)
pipeline("Hey how are you doing today?")

This code generates the following response: 
[{'generated_text': 'Hey how are you doing today? My name is David and I’m a photographer bas…
2024-10-31T03:55:28Z
Thank you for replying.  Could you help me find the instruct model that works as a chatbot please.
2024-10-31T04:14:18Z
Okay.Official one (gated. requires authorization)huggingface.cometa-llama/Llama-3.2-3B-Instruct · Hugging FaceWe’re on a journey to advance and democratize artificial intelligence through open source and open science.Unofficial ones (ungated)huggingface.counsloth/Llama-3.2-3B-Instruct · Hugging FaceWe’re on a journey to advance and democratize artificial intelligence through open source and open science.huggingface.cochuanli11/Llama-3.2-3B-Instruct-uncensored · Hugging FaceWe’re on a journey to advance and democratize artificial intelligence through open source and open science.GGUF files for Llamacpphuggingface.cobartowski/Llama-3.2-3B-Instruct-GGUF · Hugging FaceWe’re on a journey to advance and democratize artificial intelligence through open source and open science.
2024-10-31T06:45:49Z
Thank you! I used unsloth and bartowski but no luck. It gives me the same kind of weird output.
2024-10-31T06:53:26Z
How about this?huggingface.coLlama 3.2 3B Instruct - a Hugging Face Space by huggingface-projectsChatbothuggingface.coLlama 3.2 3B Uncensored Chat - a Hugging Face Space by chuanli11Discover amazing ML apps made by the community
2024-10-31T07:51:29Z
Thank you so much@John6666for taking your time and helping me out. I realized where I went wrong, I had to include a template in my model file before running it in ollama. Your first comment where you mentioned a link to a similar problem helped me out, I found this link in the comments…github.comollama/ollama/blob/main/docs/modelfile.md#valid-parameters-and-values# Ollama Model File

> [!NOTE]
> `Modelfile` syntax is in development

A model file is the blueprint to create and share models with Ollama.

## Table of Contents

- [Format](#format)
- [Examples](#examples)
- [Instructions](#instructions)
  - [FROM (Required)](#from-required)
    - [Build from existing model](#build-from-existing-model)
    - [Build from a Safetensors model](#build-from-a-safetensors-model)
    - [Build from a GGUF file](#build-from-a-gguf-file)
  - [PARAMETER](#parameter)
    - [Valid Parameters and Values](#valid-parameters-and-values)
  - [TEMPLATE](#template)
    - [Template Variables](#template-variables)This file has been truncated.show originaland it said I had to include this template in my modelfileTEMPLATE “”“{{ if .System }}<|im_start|>system{{ .System }}<|im_end|>{{ end }}{{ if .Prompt }}<|im_start|>user{{ .Prompt }}<|im_end|>{{ end }}<|im_start|>assistant“””This solved the problem and now my models work like charm. Big thanks to you for helping me out!
2024-10-31T08:11:44Z
So it was a template issue.Good to have it resolved!
2024-10-31T08:31:41Z
Yuppp, Thank you
2024-10-31T20:31:52Z
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.