metadata
library_name: transformers
tags:
- agriculture
- question-answering
- LoRA
- tinyllama
- fine-tuned
- causal-lm
license: apache-2.0
🌾 AgriQA-TinyLlama-LoRA (Adapter)
A LoRA fine-tuned TinyLlama model for answering agriculture-related questions in a conversational format. This adapter is fine-tuned on the AgriQA dataset using parameter-efficient fine-tuning (PEFT) and is suitable for low-resource inference scenarios.
🧠 Model Details
- Base Model: TinyLlama/TinyLlama-1.1B-Chat
- LoRA Adapter Size: ~2MB
- Dataset: shchoi83/agriQA
- Task: Question Answering (Instruction Tuning)
- Language: English
- Adapter Only: This repository only contains the LoRA adapter. You must load it on top of the base model.
- Trained by: @theone049
🚀 Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel, PeftConfig
# Load base model and tokenizer
base_model = AutoModelForCausalLM.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat")
tokenizer = AutoTokenizer.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat")
# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "theone049/agriqa-tinyllama-lora-adapter")
# Inference
prompt = """### Instruction:
Answer the agricultural question.
### Input:
What is the control measure for aphid infestation in mustard crops?
### Response:
"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
📊 Training
- Epochs: 3
- Batch Size: 8
- Learning Rate: 2e-5
- Precision: bf16
- Training Framework: 🤗
transformers
+peft
- Compute: Google Colab T4 GPU
📁 Files
adapter_config.json
: Configuration of LoRA adapter.adapter_model.safetensors
: The trained adapter weights.README.md
: This file.
🛑 Limitations
- Domain-Specific: Works best on agri-related questions. Not suited for general conversation.
- Small Dataset: Initial training was done on a subset (~5k samples). Accuracy may improve with full dataset.
- Not Merged: Requires base model for usage.
📚 Citation
@misc{nithyanandam2024agriqa,
title={AgriQA TinyLlama LoRA Adapter},
author={Nithyanandam Venu},
year={2024},
howpublished={\url{https://huggingface.co/theone049/agriqa-tinyllama-lora-adapter}}
}
✉️ Contact
For questions or collaborations: [email protected]
This is part of an experimental project to improve AI Q&A for agriculture. Not for medical or pesticide advice.