Qwen2-7B-Instruct-SPPO-Function-call-v2.4
This model is a fine-tuned version of slm-research-vn/Qwen2-7B-Instruct-SPPO-Function-call-v2.1 on the slm-research-vn/dpo-format-function-calling-v4, the slm-research-vn/dpo-format-glaive-code-assistant-v3-with-mistral-large-slm-iter4 and the argilla/dpo-mix-7k datasets. It achieves the following results on the evaluation set:
- Loss: 0.3152
- Rewards/chosen: 1.9961
- Rewards/rejected: 0.2161
- Rewards/accuracies: 0.8815
- Rewards/margins: 1.7800
- Logps/rejected: -267.1725
- Logps/chosen: -202.5304
- Logits/rejected: -0.6205
- Logits/chosen: -0.6185
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-06
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- distributed_type: multi-GPU
- num_devices: 8
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- total_eval_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 1
Training results
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.6534 | 0.1020 | 100 | 0.6139 | 0.2871 | 0.0961 | 0.7572 | 0.1911 | -269.5727 | -236.7095 | -0.6762 | -0.6844 |
| 0.4902 | 0.2041 | 200 | 0.4530 | 1.4421 | 0.5735 | 0.8064 | 0.8685 | -260.0234 | -213.6108 | -0.6513 | -0.6502 |
| 0.391 | 0.3061 | 300 | 0.3935 | 1.9109 | 0.6931 | 0.8382 | 1.2178 | -257.6317 | -204.2344 | -0.6321 | -0.6298 |
| 0.3497 | 0.4082 | 400 | 0.3633 | 1.9715 | 0.5740 | 0.8468 | 1.3975 | -260.0141 | -203.0221 | -0.6323 | -0.6313 |
| 0.3378 | 0.5102 | 500 | 0.3421 | 2.0346 | 0.4602 | 0.8699 | 1.5744 | -262.2907 | -201.7610 | -0.6197 | -0.6103 |
| 0.2904 | 0.6122 | 600 | 0.3287 | 1.9449 | 0.3083 | 0.8757 | 1.6366 | -265.3278 | -203.5543 | -0.6221 | -0.6159 |
| 0.3053 | 0.7143 | 700 | 0.3207 | 1.9933 | 0.2606 | 0.8902 | 1.7327 | -266.2818 | -202.5857 | -0.6162 | -0.6111 |
| 0.2655 | 0.8163 | 800 | 0.3158 | 1.9845 | 0.2262 | 0.8815 | 1.7583 | -266.9698 | -202.7614 | -0.6127 | -0.6026 |
| 0.2943 | 0.9184 | 900 | 0.3144 | 1.9968 | 0.2178 | 0.8844 | 1.7789 | -267.1377 | -202.5171 | -0.6136 | -0.6052 |
Framework versions
- PEFT 0.12.0
- Transformers 4.44.0
- Pytorch 2.3.1+cu121
- Datasets 2.20.0
- Tokenizers 0.19.1
- Downloads last month
- 14
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model has no pipeline_tag.