theone049's picture
Upload model
0181f8b verified
|
raw
history blame
2.9 kB
metadata
library_name: transformers
tags:
  - agriculture
  - question-answering
  - LoRA
  - tinyllama
  - fine-tuned
  - causal-lm
license: apache-2.0

🌾 AgriQA-TinyLlama-LoRA (Adapter)

A LoRA fine-tuned TinyLlama model for answering agriculture-related questions in a conversational format. This adapter is fine-tuned on the AgriQA dataset using parameter-efficient fine-tuning (PEFT) and is suitable for low-resource inference scenarios.

🧠 Model Details

  • Base Model: TinyLlama/TinyLlama-1.1B-Chat
  • LoRA Adapter Size: ~2MB
  • Dataset: shchoi83/agriQA
  • Task: Question Answering (Instruction Tuning)
  • Language: English
  • Adapter Only: This repository only contains the LoRA adapter. You must load it on top of the base model.
  • Trained by: @theone049

🚀 Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel, PeftConfig

# Load base model and tokenizer
base_model = AutoModelForCausalLM.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat")
tokenizer = AutoTokenizer.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat")

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "theone049/agriqa-tinyllama-lora-adapter")

# Inference
prompt = """### Instruction:
Answer the agricultural question.

### Input:
What is the control measure for aphid infestation in mustard crops?

### Response:
"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

📊 Training

  • Epochs: 3
  • Batch Size: 8
  • Learning Rate: 2e-5
  • Precision: bf16
  • Training Framework: 🤗 transformers + peft
  • Compute: Google Colab T4 GPU

📁 Files

  • adapter_config.json: Configuration of LoRA adapter.
  • adapter_model.safetensors: The trained adapter weights.
  • README.md: This file.

🛑 Limitations

  • Domain-Specific: Works best on agri-related questions. Not suited for general conversation.
  • Small Dataset: Initial training was done on a subset (~5k samples). Accuracy may improve with full dataset.
  • Not Merged: Requires base model for usage.

📚 Citation

@misc{nithyanandam2024agriqa,
  title={AgriQA TinyLlama LoRA Adapter},
  author={Nithyanandam Venu},
  year={2024},
  howpublished={\url{https://huggingface.co/theone049/agriqa-tinyllama-lora-adapter}}
}

✉️ Contact

For questions or collaborations: [email protected]


This is part of an experimental project to improve AI Q&A for agriculture. Not for medical or pesticide advice.