Hugging Face's logo

bartekupartek
/
llm2vec-qwen2.5-0.5-instruct

Safetensors
qwen2
Generated from Trainer
Eval Results
Model card Files
xet
Community
1

Improve language tag

#1
by lbourdois - opened
base: refs/heads/main
←
from: refs/pr/1
Discussion Files changed
+96
-82
Files changed (1) hide show
  1. README.md +96 -82
README.md CHANGED
@@ -1,82 +1,96 @@
1
- ---
2
- license: apache-2.0
3
- base_model: Qwen/Qwen2.5-0.5B-Instruct
4
- tags:
5
- - generated_from_trainer
6
- datasets:
7
- - wikitext
8
- metrics:
9
- - accuracy
10
- model-index:
11
- - name: llm2vec-qwen2.5-0.5-instruct
12
- results:
13
- - task:
14
- name: Masked Language Modeling
15
- type: fill-mask
16
- dataset:
17
- name: wikitext wikitext-103-raw-v1
18
- type: wikitext
19
- args: wikitext-103-raw-v1
20
- metrics:
21
- - name: Accuracy
22
- type: accuracy
23
- value: 0.629556877924779
24
- ---
25
-
26
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
27
- should probably proofread and complete it, then remove this comment. -->
28
-
29
- # llm2vec-qwen2.5-0.5-instruct
30
-
31
- This model is a fine-tuned version of [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) on the wikitext wikitext-103-raw-v1 dataset.
32
- It achieves the following results on the evaluation set:
33
- - Loss: 1.8264
34
- - Accuracy: 0.6296
35
-
36
- ## Model description
37
-
38
- More information needed
39
-
40
- ## Intended uses & limitations
41
-
42
- More information needed
43
-
44
- ## Training and evaluation data
45
-
46
- More information needed
47
-
48
- ## Training procedure
49
-
50
- ### Training hyperparameters
51
-
52
- The following hyperparameters were used during training:
53
- - learning_rate: 5e-05
54
- - train_batch_size: 16
55
- - eval_batch_size: 32
56
- - seed: 42
57
- - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
58
- - lr_scheduler_type: linear
59
- - num_epochs: 3.0
60
-
61
- ### Training results
62
-
63
- | Training Loss | Epoch | Step | Validation Loss | Accuracy |
64
- |:-------------:|:------:|:----:|:---------------:|:--------:|
65
- | No log | 0.0083 | 100 | 2.3376 | 0.5511 |
66
- | No log | 0.0166 | 200 | 2.1736 | 0.5765 |
67
- | No log | 0.0248 | 300 | 2.0679 | 0.5930 |
68
- | No log | 0.0331 | 400 | 1.9839 | 0.6056 |
69
- | 2.2761 | 0.0414 | 500 | 1.9611 | 0.6085 |
70
- | 2.2761 | 0.0497 | 600 | 1.9054 | 0.6203 |
71
- | 2.2761 | 0.0580 | 700 | 1.8838 | 0.6242 |
72
- | 2.2761 | 0.0662 | 800 | 1.8403 | 0.6296 |
73
- | 2.2761 | 0.0745 | 900 | 1.8235 | 0.6300 |
74
- | 1.8887 | 0.0828 | 1000 | 1.7920 | 0.6351 |
75
-
76
-
77
- ### Framework versions
78
-
79
- - Transformers 4.40.2
80
- - Pytorch 2.4.1+cu121
81
- - Datasets 3.0.0
82
- - Tokenizers 0.19.1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: Qwen/Qwen2.5-0.5B-Instruct
4
+ tags:
5
+ - generated_from_trainer
6
+ datasets:
7
+ - wikitext
8
+ metrics:
9
+ - accuracy
10
+ language:
11
+ - zho
12
+ - eng
13
+ - fra
14
+ - spa
15
+ - por
16
+ - deu
17
+ - ita
18
+ - rus
19
+ - jpn
20
+ - kor
21
+ - vie
22
+ - tha
23
+ - ara
24
+ model-index:
25
+ - name: llm2vec-qwen2.5-0.5-instruct
26
+ results:
27
+ - task:
28
+ type: fill-mask
29
+ name: Masked Language Modeling
30
+ dataset:
31
+ name: wikitext wikitext-103-raw-v1
32
+ type: wikitext
33
+ args: wikitext-103-raw-v1
34
+ metrics:
35
+ - type: accuracy
36
+ value: 0.629556877924779
37
+ name: Accuracy
38
+ ---
39
+
40
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
41
+ should probably proofread and complete it, then remove this comment. -->
42
+
43
+ # llm2vec-qwen2.5-0.5-instruct
44
+
45
+ This model is a fine-tuned version of [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) on the wikitext wikitext-103-raw-v1 dataset.
46
+ It achieves the following results on the evaluation set:
47
+ - Loss: 1.8264
48
+ - Accuracy: 0.6296
49
+
50
+ ## Model description
51
+
52
+ More information needed
53
+
54
+ ## Intended uses & limitations
55
+
56
+ More information needed
57
+
58
+ ## Training and evaluation data
59
+
60
+ More information needed
61
+
62
+ ## Training procedure
63
+
64
+ ### Training hyperparameters
65
+
66
+ The following hyperparameters were used during training:
67
+ - learning_rate: 5e-05
68
+ - train_batch_size: 16
69
+ - eval_batch_size: 32
70
+ - seed: 42
71
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
72
+ - lr_scheduler_type: linear
73
+ - num_epochs: 3.0
74
+
75
+ ### Training results
76
+
77
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy |
78
+ |:-------------:|:------:|:----:|:---------------:|:--------:|
79
+ | No log | 0.0083 | 100 | 2.3376 | 0.5511 |
80
+ | No log | 0.0166 | 200 | 2.1736 | 0.5765 |
81
+ | No log | 0.0248 | 300 | 2.0679 | 0.5930 |
82
+ | No log | 0.0331 | 400 | 1.9839 | 0.6056 |
83
+ | 2.2761 | 0.0414 | 500 | 1.9611 | 0.6085 |
84
+ | 2.2761 | 0.0497 | 600 | 1.9054 | 0.6203 |
85
+ | 2.2761 | 0.0580 | 700 | 1.8838 | 0.6242 |
86
+ | 2.2761 | 0.0662 | 800 | 1.8403 | 0.6296 |
87
+ | 2.2761 | 0.0745 | 900 | 1.8235 | 0.6300 |
88
+ | 1.8887 | 0.0828 | 1000 | 1.7920 | 0.6351 |
89
+
90
+
91
+ ### Framework versions
92
+
93
+ - Transformers 4.40.2
94
+ - Pytorch 2.4.1+cu121
95
+ - Datasets 3.0.0
96
+ - Tokenizers 0.19.1