|
|
---
|
|
|
library_name: transformers
|
|
|
tags:
|
|
|
- legal-qa
|
|
|
- lora
|
|
|
- mistral-7b
|
|
|
- huggingface
|
|
|
- langchain
|
|
|
- faiss
|
|
|
- legal-tech
|
|
|
- retrieval-augmented-generation
|
|
|
datasets:
|
|
|
- theatticusproject/cuad
|
|
|
- jhu-clsp/SARA
|
|
|
language:
|
|
|
- en
|
|
|
base_model:
|
|
|
- mistralai/Mistral-7B-Instruct-v0.2
|
|
|
---
|
|
|
|
|
|
# 🧠 LexiQ – Legal Document QA using LoRA-Finetuned Mistral-7B |
|
|
|
|
|
> **Authors**: Nishad Kookana, Ishita, Milan Agarwal, and Sushant Bhatia |
|
|
> **License**: Apache 2.0 |
|
|
> **Base model**: [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) |
|
|
|
|
|
LexiQ is a domain-specific Legal Question-Answering model built on top of Mistral 7B. It is fine-tuned using **LoRA** on two legal datasets: **CUAD** (Contract Understanding Atticus Dataset) and **SARA** (Summarization of Appellate Court Rulings). The model supports question answering from legal documents with Retrieval-Augmented Generation (RAG). |
|
|
|
|
|
--- |
|
|
|
|
|
## 🔍 Use Cases |
|
|
|
|
|
### ✅ Direct Use: |
|
|
- Legal document understanding |
|
|
- Question answering over case files, legal summaries, contracts |
|
|
- Legal education, research, and policy drafting |
|
|
|
|
|
### ❌ Out-of-Scope Use: |
|
|
- Not suitable for non-English documents |
|
|
- Not trained on international law — Indian and US law concepts dominate |
|
|
- Not intended for legal advice in real-world litigation |
|
|
|
|
|
--- |
|
|
|
|
|
## 🧪 Model Details |
|
|
|
|
|
- **Model Type**: Causal Language Model (LLM) with LoRA fine-tuning |
|
|
- **Languages**: English |
|
|
- **Quantization**: 4-bit (`bnb` nf4) via `bitsandbytes` |
|
|
- **Training Precision**: Mixed FP16 |
|
|
- **Training Datasets**: CUAD, SARA |
|
|
- **Tokenizer**: AutoTokenizer (fast) |
|
|
- **Training Framework**: Transformers + PEFT + Accelerate |
|
|
|
|
|
--- |
|
|
|
|
|
## 📦 How to Use |
|
|
|
|
|
```python |
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM |
|
|
from peft import PeftModel |
|
|
import torch |
|
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained("NishKook/legal-qa-lora") |
|
|
base_model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2", device_map="auto", torch_dtype=torch.float16) |
|
|
model = PeftModel.from_pretrained(base_model, "NishKook/legal-qa-lora", device_map="auto") |
|
|
|
|
|
question = "What are the four elements of negligence?" |
|
|
context = "Negligence requires a duty, a breach of that duty, causation, and damages." |
|
|
|
|
|
prompt = f"### Question:\n{question}\n\n### Context:\n{context}\n\n### Answer:\n" |
|
|
inputs = tokenizer(prompt, return_tensors="pt").to("cuda") |
|
|
outputs = model.generate(**inputs, max_new_tokens=256) |
|
|
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)) |
