banty1614/codeforge-qwen-lora

🚀 CodeForge AI

Fine-tuned Qwen2.5-3B-Instruct using QLoRA + PEFT for software engineering tasks.

🚀 CodeForge AI

A QLoRA fine-tuned version of Qwen2.5-3B-Instruct for software engineering and Python coding tasks.

---

📖 Overview

CodeForge AI is a parameter-efficient fine-tuned (PEFT) Large Language Model built on Qwen2.5-3B-Instruct.

The objective of this project is to demonstrate a complete end-to-end modern LLM fine-tuning workflow using:

• QLoRA
• LoRA
• PEFT
• Hugging Face Transformers
• TRL SFTTrainer
• Google Colab Tesla T4

Instead of updating all 3.1 billion parameters, only 0.96% of the parameters were trained, making the project lightweight and GPU-efficient.

---

🏗 Base Model

Model

Qwen/Qwen2.5-3B-Instruct

---

⚡ Fine-Tuning Technique

• QLoRA (4-bit Quantization)
• LoRA (Low-Rank Adaptation)
• PEFT (Parameter Efficient Fine Tuning)
• Supervised Fine-Tuning (SFT)

---

📚 Dataset

Dataset used:

iamtarun/python_code_instructions_18k_alpaca

Dataset contains approximately 18,000 Python instruction-following examples.

For this proof-of-concept training run, a subset of the dataset was used.

---

⚙ Training Configuration

Parameter	Value
Base Model	Qwen2.5-3B-Instruct
Fine-Tuning	QLoRA
LoRA Rank (r)	16
Alpha	32
Dropout	0.05
Quantization	4-bit NF4
Optimizer	AdamW
Learning Rate	2e-4
Max Steps	100
Batch Size	2
Gradient Accumulation	4
Max Sequence Length	512
GPU	Tesla T4 (16 GB)

---

📊 Training Results

Training completed successfully.

Final Training Loss:

0.7047

Trainable Parameters:

29,933,568

Total Parameters:

3,115,872,256

Percentage of Trainable Parameters:

0.9607%

---

🧠 Architecture

Python Instruction Dataset
            │
            ▼
      Tokenization
            │
            ▼
Qwen2.5-3B-Instruct
            │
            ▼
    4-bit Quantization
            │
            ▼
          QLoRA
            │
            ▼
      LoRA Adapters
            │
            ▼
     Supervised Fine-Tuning
            │
            ▼
     Fine-Tuned Adapter
            │
            ▼
      Hugging Face Hub

---

💻 Usage

from transformers import AutoTokenizer
from peft import AutoPeftModelForCausalLM

model = AutoPeftModelForCausalLM.from_pretrained(
    "banty1614/codeforge-qwen-lora",
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained(
    "banty1614/codeforge-qwen-lora"
)

Example:

messages = [
    {
        "role": "user",
        "content": "Write a Python function to reverse a linked list."
    }
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

---

📈 Evaluation

The model was compared with the original base model on multiple Python programming prompts.

Evaluation included:

• Code correctness
• Explanation quality
• Readability
• Response formatting

This project demonstrates the complete QLoRA fine-tuning workflow rather than claiming to outperform the original Qwen model across all programming tasks.

---

⚠ Limitations

• Trained for only 100 optimization steps.
• Uses a subset of the original dataset.
• Intended as a proof-of-concept for parameter-efficient fine-tuning.
• Not evaluated on standardized coding benchmarks.

---

🚀 Future Work

• Train on the complete dataset.
• Increase training epochs.
• Add evaluation on HumanEval and MBPP.
• Expand training to FastAPI, SQL, Docker, Git, and System Design.
• Support multiple LoRA adapters for different software engineering domains.

---

🙏 Acknowledgements

• Alibaba Cloud for Qwen2.5
• Hugging Face
• TRL
• PEFT
• BitsAndBytes
• Google Colab

---

If you use this project for learning or research, please consider giving the repository a ⭐.