Llama-instruct

This model is a fine-tuned version of the Llama3.2-3B model optimized for instruciton following.

Fine-tuning dataset

Model was fine-tuned on tatsu-lab/alpaca instruction dataset.

Inference

!pip install torch transformers accelerate bitsandbytes

from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline

model_name = "entfane/llama-instruct"

tokenizer = AutoTokenizer.from_pretrained(model_name)

# Load model in 4-bit precision
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    load_in_4bit=True,
    device_map="auto"
)

pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer
)

instruction = "Give me some health tips"
prompt = f"### Instruction:\n{instruction}\n\n### Response:"

output = pipe(prompt)
print(output[0]['generated_text'])