Hugging Face's logo

vikasp1610
/
qwen2-sql-lora

Text Generation
PEFT
Safetensors
Transformers
lora
conversational
Model card Files Community
qwen2-sql-lora / README.md
vikasp1610's picture
vikasp1610
End of training
16182d9 verified
preview code
|
raw
history blame contribute delete
1.8 kB
metadata
library_name: peft
license: apache-2.0
base_model: Qwen/Qwen2-0.5B
tags:
  - base_model:adapter:Qwen/Qwen2-0.5B
  - lora
  - transformers
pipeline_tag: text-generation
model-index:
  - name: qwen2-sql-lora
    results: []

qwen2-sql-lora

This model is a fine-tuned version of Qwen/Qwen2-0.5B on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.6407

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0002
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 16
  • optimizer: Use adamw_torch_fused with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • num_epochs: 2
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
0.6799 0.3282 500 0.6805
0.6903 0.6564 1000 0.6626
0.6142 0.9846 1500 0.6526
0.6491 1.3124 2000 0.6463
0.6526 1.6406 2500 0.6428
0.6208 1.9688 3000 0.6407

Framework versions

  • PEFT 0.17.0
  • Transformers 4.55.2
  • Pytorch 2.8.0+cu128
  • Datasets 4.0.0
  • Tokenizers 0.21.4