Hugging Face's logo

henggg
/
ReSo

Safetensors
Model card Files Community
henggg commited on
Commit
fc88c06
·
verified ·
1 Parent(s): b9c1a11

Upload 5 files

Browse files
Files changed (5) hide show
  1. CRM/sft-qwen2.5-math-prm-7b-score-model-simple-bs128/README.md +202 -0
  2. CRM/sft-qwen2.5-math-prm-7b-score-model-simple-bs128/adapter_config.json +31 -0
  3. CRM/sft-qwen2.5-math-prm-7b-score-model-simple-bs128/adapter_model.safetensors +3 -0
  4. CRM/sft-qwen2.5-math-prm-7b-score-model-simple-bs128/trainer_state.json +3084 -0
  5. CRM/sft-qwen2.5-math-prm-7b-score-model-simple-bs128/training_args.bin +3 -0
CRM/sft-qwen2.5-math-prm-7b-score-model-simple-bs128/README.md ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: /home/xuexiangyuan/workspace/score-model/model
3
+ library_name: peft
4
+ ---
5
+
6
+ # Model Card for Model ID
7
+
8
+ <!-- Provide a quick summary of what the model is/does. -->
9
+
10
+
11
+
12
+ ## Model Details
13
+
14
+ ### Model Description
15
+
16
+ <!-- Provide a longer summary of what this model is. -->
17
+
18
+
19
+
20
+ - **Developed by:** [More Information Needed]
21
+ - **Funded by [optional]:** [More Information Needed]
22
+ - **Shared by [optional]:** [More Information Needed]
23
+ - **Model type:** [More Information Needed]
24
+ - **Language(s) (NLP):** [More Information Needed]
25
+ - **License:** [More Information Needed]
26
+ - **Finetuned from model [optional]:** [More Information Needed]
27
+
28
+ ### Model Sources [optional]
29
+
30
+ <!-- Provide the basic links for the model. -->
31
+
32
+ - **Repository:** [More Information Needed]
33
+ - **Paper [optional]:** [More Information Needed]
34
+ - **Demo [optional]:** [More Information Needed]
35
+
36
+ ## Uses
37
+
38
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
+
40
+ ### Direct Use
41
+
42
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
+
44
+ [More Information Needed]
45
+
46
+ ### Downstream Use [optional]
47
+
48
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
+
50
+ [More Information Needed]
51
+
52
+ ### Out-of-Scope Use
53
+
54
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
+
56
+ [More Information Needed]
57
+
58
+ ## Bias, Risks, and Limitations
59
+
60
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
+
62
+ [More Information Needed]
63
+
64
+ ### Recommendations
65
+
66
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
+
68
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
+
70
+ ## How to Get Started with the Model
71
+
72
+ Use the code below to get started with the model.
73
+
74
+ [More Information Needed]
75
+
76
+ ## Training Details
77
+
78
+ ### Training Data
79
+
80
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
+
82
+ [More Information Needed]
83
+
84
+ ### Training Procedure
85
+
86
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
+
88
+ #### Preprocessing [optional]
89
+
90
+ [More Information Needed]
91
+
92
+
93
+ #### Training Hyperparameters
94
+
95
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
+
97
+ #### Speeds, Sizes, Times [optional]
98
+
99
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
+
101
+ [More Information Needed]
102
+
103
+ ## Evaluation
104
+
105
+ <!-- This section describes the evaluation protocols and provides the results. -->
106
+
107
+ ### Testing Data, Factors & Metrics
108
+
109
+ #### Testing Data
110
+
111
+ <!-- This should link to a Dataset Card if possible. -->
112
+
113
+ [More Information Needed]
114
+
115
+ #### Factors
116
+
117
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
+
119
+ [More Information Needed]
120
+
121
+ #### Metrics
122
+
123
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
+
125
+ [More Information Needed]
126
+
127
+ ### Results
128
+
129
+ [More Information Needed]
130
+
131
+ #### Summary
132
+
133
+
134
+
135
+ ## Model Examination [optional]
136
+
137
+ <!-- Relevant interpretability work for the model goes here -->
138
+
139
+ [More Information Needed]
140
+
141
+ ## Environmental Impact
142
+
143
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
+
145
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
+
147
+ - **Hardware Type:** [More Information Needed]
148
+ - **Hours used:** [More Information Needed]
149
+ - **Cloud Provider:** [More Information Needed]
150
+ - **Compute Region:** [More Information Needed]
151
+ - **Carbon Emitted:** [More Information Needed]
152
+
153
+ ## Technical Specifications [optional]
154
+
155
+ ### Model Architecture and Objective
156
+
157
+ [More Information Needed]
158
+
159
+ ### Compute Infrastructure
160
+
161
+ [More Information Needed]
162
+
163
+ #### Hardware
164
+
165
+ [More Information Needed]
166
+
167
+ #### Software
168
+
169
+ [More Information Needed]
170
+
171
+ ## Citation [optional]
172
+
173
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
+
175
+ **BibTeX:**
176
+
177
+ [More Information Needed]
178
+
179
+ **APA:**
180
+
181
+ [More Information Needed]
182
+
183
+ ## Glossary [optional]
184
+
185
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
+
187
+ [More Information Needed]
188
+
189
+ ## More Information [optional]
190
+
191
+ [More Information Needed]
192
+
193
+ ## Model Card Authors [optional]
194
+
195
+ [More Information Needed]
196
+
197
+ ## Model Card Contact
198
+
199
+ [More Information Needed]
200
+ ### Framework versions
201
+
202
+ - PEFT 0.12.0
CRM/sft-qwen2.5-math-prm-7b-score-model-simple-bs128/adapter_config.json ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "/home/xuexiangyuan/workspace/score-model/model",
5
+ "bias": "none",
6
+ "fan_in_fan_out": false,
7
+ "inference_mode": true,
8
+ "init_lora_weights": true,
9
+ "layer_replication": null,
10
+ "layers_pattern": null,
11
+ "layers_to_transform": null,
12
+ "loftq_config": {},
13
+ "lora_alpha": 32.0,
14
+ "lora_dropout": 0.1,
15
+ "megatron_config": null,
16
+ "megatron_core": "megatron.core",
17
+ "modules_to_save": null,
18
+ "peft_type": "LORA",
19
+ "r": 8,
20
+ "rank_pattern": {},
21
+ "revision": null,
22
+ "target_modules": [
23
+ "q_proj",
24
+ "v_proj",
25
+ "score.2",
26
+ "score.0"
27
+ ],
28
+ "task_type": "CAUSAL_LM",
29
+ "use_dora": false,
30
+ "use_rslora": false
31
+ }
CRM/sft-qwen2.5-math-prm-7b-score-model-simple-bs128/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e16e930b3fc6cc8c77aa418990335b7783354ce64ab4045c02b17578d007dbef
3
+ size 10451840
CRM/sft-qwen2.5-math-prm-7b-score-model-simple-bs128/trainer_state.json ADDED
@@ -0,0 +1,3084 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 9.768,
5
+ "eval_steps": 10,
6
+ "global_step": 390,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.0256,
13
+ "grad_norm": 0.6148048043251038,
14
+ "learning_rate": 9.974358974358975e-05,
15
+ "loss": 0.645,
16
+ "step": 1
17
+ },
18
+ {
19
+ "epoch": 0.0512,
20
+ "grad_norm": 1.038170337677002,
21
+ "learning_rate": 9.948717948717949e-05,
22
+ "loss": 0.6743,
23
+ "step": 2
24
+ },
25
+ {
26
+ "epoch": 0.0768,
27
+ "grad_norm": 0.6856626868247986,
28
+ "learning_rate": 9.923076923076923e-05,
29
+ "loss": 0.6746,
30
+ "step": 3
31
+ },
32
+ {
33
+ "epoch": 0.1024,
34
+ "grad_norm": 1.1796700954437256,
35
+ "learning_rate": 9.897435897435898e-05,
36
+ "loss": 0.6025,
37
+ "step": 4
38
+ },
39
+ {
40
+ "epoch": 0.128,
41
+ "grad_norm": 0.8707571625709534,
42
+ "learning_rate": 9.871794871794872e-05,
43
+ "loss": 0.6289,
44
+ "step": 5
45
+ },
46
+ {
47
+ "epoch": 0.1536,
48
+ "grad_norm": 2.7378222942352295,
49
+ "learning_rate": 9.846153846153848e-05,
50
+ "loss": 0.6433,
51
+ "step": 6
52
+ },
53
+ {
54
+ "epoch": 0.1792,
55
+ "grad_norm": 4.110616683959961,
56
+ "learning_rate": 9.820512820512821e-05,
57
+ "loss": 0.6826,
58
+ "step": 7
59
+ },
60
+ {
61
+ "epoch": 0.2048,
62
+ "grad_norm": 2.4638514518737793,
63
+ "learning_rate": 9.794871794871795e-05,
64
+ "loss": 0.6047,
65
+ "step": 8
66
+ },
67
+ {
68
+ "epoch": 0.2304,
69
+ "grad_norm": 2.8323960304260254,
70
+ "learning_rate": 9.76923076923077e-05,
71
+ "loss": 0.5871,
72
+ "step": 9
73
+ },
74
+ {
75
+ "epoch": 0.256,
76
+ "grad_norm": 2.379040479660034,
77
+ "learning_rate": 9.743589743589744e-05,
78
+ "loss": 0.5875,
79
+ "step": 10
80
+ },
81
+ {
82
+ "epoch": 0.256,
83
+ "eval_loss": 0.5159781575202942,
84
+ "eval_runtime": 46.1024,
85
+ "eval_samples_per_second": 9.783,
86
+ "eval_steps_per_second": 0.174,
87
+ "step": 10
88
+ },
89
+ {
90
+ "epoch": 0.2816,
91
+ "grad_norm": 1.054911732673645,
92
+ "learning_rate": 9.717948717948718e-05,
93
+ "loss": 0.531,
94
+ "step": 11
95
+ },
96
+ {
97
+ "epoch": 0.3072,
98
+ "grad_norm": 1.300252079963684,
99
+ "learning_rate": 9.692307692307692e-05,
100
+ "loss": 0.5179,
101
+ "step": 12
102
+ },
103
+ {
104
+ "epoch": 0.3328,
105
+ "grad_norm": 2.310645341873169,
106
+ "learning_rate": 9.666666666666667e-05,
107
+ "loss": 0.5082,
108
+ "step": 13
109
+ },
110
+ {
111
+ "epoch": 0.3584,
112
+ "grad_norm": 1.4785258769989014,
113
+ "learning_rate": 9.641025641025641e-05,
114
+ "loss": 0.465,
115
+ "step": 14
116
+ },
117
+ {
118
+ "epoch": 0.384,
119
+ "grad_norm": 1.6794428825378418,
120
+ "learning_rate": 9.615384615384617e-05,
121
+ "loss": 0.4115,
122
+ "step": 15
123
+ },
124
+ {
125
+ "epoch": 0.4096,
126
+ "grad_norm": 1.2971839904785156,
127
+ "learning_rate": 9.589743589743591e-05,
128
+ "loss": 0.4262,
129
+ "step": 16
130
+ },
131
+ {
132
+ "epoch": 0.4352,
133
+ "grad_norm": 1.181352972984314,
134
+ "learning_rate": 9.564102564102565e-05,
135
+ "loss": 0.43,
136
+ "step": 17
137
+ },
138
+ {
139
+ "epoch": 0.4608,
140
+ "grad_norm": 1.1496334075927734,
141
+ "learning_rate": 9.53846153846154e-05,
142
+ "loss": 0.3859,
143
+ "step": 18
144
+ },
145
+ {
146
+ "epoch": 0.4864,
147
+ "grad_norm": 2.361189842224121,
148
+ "learning_rate": 9.512820512820513e-05,
149
+ "loss": 0.4587,
150
+ "step": 19
151
+ },
152
+ {
153
+ "epoch": 0.512,
154
+ "grad_norm": 2.7645130157470703,
155
+ "learning_rate": 9.487179487179487e-05,
156
+ "loss": 0.3971,
157
+ "step": 20
158
+ },
159
+ {
160
+ "epoch": 0.512,
161
+ "eval_loss": 0.30324116349220276,
162
+ "eval_runtime": 46.0919,
163
+ "eval_samples_per_second": 9.785,
164
+ "eval_steps_per_second": 0.174,
165
+ "step": 20
166
+ },
167
+ {
168
+ "epoch": 0.5376,
169
+ "grad_norm": 1.1214560270309448,
170
+ "learning_rate": 9.461538461538461e-05,
171
+ "loss": 0.3439,
172
+ "step": 21
173
+ },
174
+ {
175
+ "epoch": 0.5632,
176
+ "grad_norm": 1.79205322265625,
177
+ "learning_rate": 9.435897435897436e-05,
178
+ "loss": 0.3618,
179
+ "step": 22
180
+ },
181
+ {
182
+ "epoch": 0.5888,
183
+ "grad_norm": 1.568602442741394,
184
+ "learning_rate": 9.41025641025641e-05,
185
+ "loss": 0.4483,
186
+ "step": 23
187
+ },
188
+ {
189
+ "epoch": 0.6144,
190
+ "grad_norm": 1.2847480773925781,
191
+ "learning_rate": 9.384615384615386e-05,
192
+ "loss": 0.3828,
193
+ "step": 24
194
+ },
195
+ {
196
+ "epoch": 0.64,
197
+ "grad_norm": 1.417415976524353,
198
+ "learning_rate": 9.35897435897436e-05,
199
+ "loss": 0.4042,
200
+ "step": 25
201
+ },
202
+ {
203
+ "epoch": 0.6656,
204
+ "grad_norm": 2.1015641689300537,
205
+ "learning_rate": 9.333333333333334e-05,
206
+ "loss": 0.3737,
207
+ "step": 26
208
+ },
209
+ {
210
+ "epoch": 0.6912,
211
+ "grad_norm": 2.055237293243408,
212
+ "learning_rate": 9.307692307692309e-05,
213
+ "loss": 0.3245,
214
+ "step": 27
215
+ },
216
+ {
217
+ "epoch": 0.7168,
218
+ "grad_norm": 1.224246859550476,
219
+ "learning_rate": 9.282051282051283e-05,
220
+ "loss": 0.3457,
221
+ "step": 28
222
+ },
223
+ {
224
+ "epoch": 0.7424,
225
+ "grad_norm": 1.686336636543274,
226
+ "learning_rate": 9.256410256410257e-05,
227
+ "loss": 0.2515,
228
+ "step": 29
229
+ },
230
+ {
231
+ "epoch": 0.768,
232
+ "grad_norm": 0.8864126205444336,
233
+ "learning_rate": 9.230769230769232e-05,
234
+ "loss": 0.318,
235
+ "step": 30
236
+ },
237
+ {
238
+ "epoch": 0.768,
239
+ "eval_loss": 0.2457895278930664,
240
+ "eval_runtime": 46.2363,
241
+ "eval_samples_per_second": 9.754,
242
+ "eval_steps_per_second": 0.173,
243
+ "step": 30
244
+ },
245
+ {
246
+ "epoch": 0.7936,
247
+ "grad_norm": 1.4351671934127808,
248
+ "learning_rate": 9.205128205128205e-05,
249
+ "loss": 0.414,
250
+ "step": 31
251
+ },
252
+ {
253
+ "epoch": 0.8192,
254
+ "grad_norm": 1.2227267026901245,
255
+ "learning_rate": 9.179487179487179e-05,
256
+ "loss": 0.2987,
257
+ "step": 32
258
+ },
259
+ {
260
+ "epoch": 0.8448,
261
+ "grad_norm": 1.0370402336120605,
262
+ "learning_rate": 9.153846153846155e-05,
263
+ "loss": 0.3175,
264
+ "step": 33
265
+ },
266
+ {
267
+ "epoch": 0.8704,
268
+ "grad_norm": 1.1455687284469604,
269
+ "learning_rate": 9.128205128205129e-05,
270
+ "loss": 0.3664,
271
+ "step": 34
272
+ },
273
+ {
274
+ "epoch": 0.896,
275
+ "grad_norm": 0.978793740272522,
276
+ "learning_rate": 9.102564102564103e-05,
277
+ "loss": 0.3321,
278
+ "step": 35
279
+ },
280
+ {
281
+ "epoch": 0.9216,
282
+ "grad_norm": 1.6957523822784424,
283
+ "learning_rate": 9.076923076923078e-05,
284
+ "loss": 0.317,
285
+ "step": 36
286
+ },
287
+ {
288
+ "epoch": 0.9472,
289
+ "grad_norm": 1.6317747831344604,
290
+ "learning_rate": 9.051282051282052e-05,
291
+ "loss": 0.2248,
292
+ "step": 37
293
+ },
294
+ {
295
+ "epoch": 0.9728,
296
+ "grad_norm": 1.592618703842163,
297
+ "learning_rate": 9.025641025641026e-05,
298
+ "loss": 0.3547,
299
+ "step": 38
300
+ },
301
+ {
302
+ "epoch": 0.9984,
303
+ "grad_norm": 1.3694124221801758,
304
+ "learning_rate": 9e-05,
305
+ "loss": 0.3967,
306
+ "step": 39
307
+ },
308
+ {
309
+ "epoch": 1.0,
310
+ "grad_norm": 0.802060067653656,
311
+ "learning_rate": 8.974358974358975e-05,
312
+ "loss": 0.0485,
313
+ "step": 40
314
+ },
315
+ {
316
+ "epoch": 1.0,
317
+ "eval_loss": 0.2322917878627777,
318
+ "eval_runtime": 46.2118,
319
+ "eval_samples_per_second": 9.759,
320
+ "eval_steps_per_second": 0.173,
321
+ "step": 40
322
+ },
323
+ {
324
+ "epoch": 1.0256,
325
+ "grad_norm": 1.097848892211914,
326
+ "learning_rate": 8.948717948717949e-05,
327
+ "loss": 0.2527,
328
+ "step": 41
329
+ },
330
+ {
331
+ "epoch": 1.0512,
332
+ "grad_norm": 2.0955655574798584,
333
+ "learning_rate": 8.923076923076924e-05,
334
+ "loss": 0.385,
335
+ "step": 42
336
+ },
337
+ {
338
+ "epoch": 1.0768,
339
+ "grad_norm": 1.8478257656097412,
340
+ "learning_rate": 8.897435897435898e-05,
341
+ "loss": 0.3122,
342
+ "step": 43
343
+ },
344
+ {
345
+ "epoch": 1.1024,
346
+ "grad_norm": 0.8632456660270691,
347
+ "learning_rate": 8.871794871794872e-05,
348
+ "loss": 0.2293,
349
+ "step": 44
350
+ },
351
+ {
352
+ "epoch": 1.1280000000000001,
353
+ "grad_norm": 1.1462959051132202,
354
+ "learning_rate": 8.846153846153847e-05,
355
+ "loss": 0.2703,
356
+ "step": 45
357
+ },
358
+ {
359
+ "epoch": 1.1536,
360
+ "grad_norm": 1.3070207834243774,
361
+ "learning_rate": 8.820512820512821e-05,
362
+ "loss": 0.3301,
363
+ "step": 46
364
+ },
365
+ {
366
+ "epoch": 1.1792,
367
+ "grad_norm": 1.0568207502365112,
368
+ "learning_rate": 8.794871794871795e-05,
369
+ "loss": 0.2951,
370
+ "step": 47
371
+ },
372
+ {
373
+ "epoch": 1.2048,
374
+ "grad_norm": 0.943413257598877,
375
+ "learning_rate": 8.76923076923077e-05,
376
+ "loss": 0.2531,
377
+ "step": 48
378
+ },
379
+ {
380
+ "epoch": 1.2304,
381
+ "grad_norm": 0.8083965182304382,
382
+ "learning_rate": 8.743589743589744e-05,
383
+ "loss": 0.2516,
384
+ "step": 49
385
+ },
386
+ {
387
+ "epoch": 1.256,
388
+ "grad_norm": 0.9122912883758545,
389
+ "learning_rate": 8.717948717948718e-05,
390
+ "loss": 0.2332,
391
+ "step": 50
392
+ },
393
+ {
394
+ "epoch": 1.256,
395
+ "eval_loss": 0.22658510506153107,
396
+ "eval_runtime": 46.128,
397
+ "eval_samples_per_second": 9.777,
398
+ "eval_steps_per_second": 0.173,
399
+ "step": 50
400
+ },
401
+ {
402
+ "epoch": 1.2816,
403
+ "grad_norm": 0.8947486877441406,
404
+ "learning_rate": 8.692307692307692e-05,
405
+ "loss": 0.2393,
406
+ "step": 51
407
+ },
408
+ {
409
+ "epoch": 1.3072,
410
+ "grad_norm": 1.822934627532959,
411
+ "learning_rate": 8.666666666666667e-05,
412
+ "loss": 0.3606,
413
+ "step": 52
414
+ },
415
+ {
416
+ "epoch": 1.3328,
417
+ "grad_norm": 1.1772205829620361,
418
+ "learning_rate": 8.641025641025642e-05,
419
+ "loss": 0.3321,
420
+ "step": 53
421
+ },
422
+ {
423
+ "epoch": 1.3584,
424
+ "grad_norm": 1.822982907295227,
425
+ "learning_rate": 8.615384615384617e-05,
426
+ "loss": 0.2437,
427
+ "step": 54
428
+ },
429
+ {
430
+ "epoch": 1.384,
431
+ "grad_norm": 2.0824837684631348,
432
+ "learning_rate": 8.58974358974359e-05,
433
+ "loss": 0.1932,
434
+ "step": 55
435
+ },
436
+ {
437
+ "epoch": 1.4096,
438
+ "grad_norm": 1.6939680576324463,
439
+ "learning_rate": 8.564102564102564e-05,
440
+ "loss": 0.3335,
441
+ "step": 56
442
+ },
443
+ {
444
+ "epoch": 1.4352,
445
+ "grad_norm": 1.8843656778335571,
446
+ "learning_rate": 8.538461538461538e-05,
447
+ "loss": 0.2614,
448
+ "step": 57
449
+ },
450
+ {
451
+ "epoch": 1.4607999999999999,
452
+ "grad_norm": 1.1397833824157715,
453
+ "learning_rate": 8.512820512820513e-05,
454
+ "loss": 0.2694,
455
+ "step": 58
456
+ },
457
+ {
458
+ "epoch": 1.4864,
459
+ "grad_norm": 1.1693910360336304,
460
+ "learning_rate": 8.487179487179487e-05,
461
+ "loss": 0.2141,
462
+ "step": 59
463
+ },
464
+ {
465
+ "epoch": 1.512,
466
+ "grad_norm": 1.0279395580291748,
467
+ "learning_rate": 8.461538461538461e-05,
468
+ "loss": 0.2065,
469
+ "step": 60
470
+ },
471
+ {
472
+ "epoch": 1.512,
473
+ "eval_loss": 0.20384371280670166,
474
+ "eval_runtime": 46.065,
475
+ "eval_samples_per_second": 9.791,
476
+ "eval_steps_per_second": 0.174,
477
+ "step": 60
478
+ },
479
+ {
480
+ "epoch": 1.5375999999999999,
481
+ "grad_norm": 0.9246665835380554,
482
+ "learning_rate": 8.435897435897436e-05,
483
+ "loss": 0.2704,
484
+ "step": 61
485
+ },
486
+ {
487
+ "epoch": 1.5632000000000001,
488
+ "grad_norm": 1.0461363792419434,
489
+ "learning_rate": 8.410256410256411e-05,
490
+ "loss": 0.1973,
491
+ "step": 62
492
+ },
493
+ {
494
+ "epoch": 1.5888,
495
+ "grad_norm": 2.1885361671447754,
496
+ "learning_rate": 8.384615384615386e-05,
497
+ "loss": 0.2158,
498
+ "step": 63
499
+ },
500
+ {
501
+ "epoch": 1.6143999999999998,
502
+ "grad_norm": 1.0617963075637817,
503
+ "learning_rate": 8.35897435897436e-05,
504
+ "loss": 0.2576,
505
+ "step": 64
506
+ },
507
+ {
508
+ "epoch": 1.6400000000000001,
509
+ "grad_norm": 1.603236198425293,
510
+ "learning_rate": 8.333333333333334e-05,
511
+ "loss": 0.2539,
512
+ "step": 65
513
+ },
514
+ {
515
+ "epoch": 1.6656,
516
+ "grad_norm": 1.3767043352127075,
517
+ "learning_rate": 8.307692307692309e-05,
518
+ "loss": 0.297,
519
+ "step": 66
520
+ },
521
+ {
522
+ "epoch": 1.6912,
523
+ "grad_norm": 1.6388436555862427,
524
+ "learning_rate": 8.282051282051283e-05,
525
+ "loss": 0.2893,
526
+ "step": 67
527
+ },
528
+ {
529
+ "epoch": 1.7168,
530
+ "grad_norm": 1.2009103298187256,
531
+ "learning_rate": 8.256410256410256e-05,
532
+ "loss": 0.2061,
533
+ "step": 68
534
+ },
535
+ {
536
+ "epoch": 1.7424,
537
+ "grad_norm": 1.3011361360549927,
538
+ "learning_rate": 8.23076923076923e-05,
539
+ "loss": 0.1979,
540
+ "step": 69
541
+ },
542
+ {
543
+ "epoch": 1.768,
544
+ "grad_norm": 1.444062352180481,
545
+ "learning_rate": 8.205128205128205e-05,
546
+ "loss": 0.2701,
547
+ "step": 70
548
+ },
549
+ {
550
+ "epoch": 1.768,
551
+ "eval_loss": 0.19174103438854218,
552
+ "eval_runtime": 46.1067,
553
+ "eval_samples_per_second": 9.782,
554
+ "eval_steps_per_second": 0.174,
555
+ "step": 70
556
+ },
557
+ {
558
+ "epoch": 1.7936,
559
+ "grad_norm": 1.275272250175476,
560
+ "learning_rate": 8.179487179487179e-05,
561
+ "loss": 0.2422,
562
+ "step": 71
563
+ },
564
+ {
565
+ "epoch": 1.8192,
566
+ "grad_norm": 1.6378053426742554,
567
+ "learning_rate": 8.153846153846155e-05,
568
+ "loss": 0.2733,
569
+ "step": 72
570
+ },
571
+ {
572
+ "epoch": 1.8448,
573
+ "grad_norm": 1.6403453350067139,
574
+ "learning_rate": 8.128205128205129e-05,
575
+ "loss": 0.3505,
576
+ "step": 73
577
+ },
578
+ {
579
+ "epoch": 1.8704,
580
+ "grad_norm": 1.606053113937378,
581
+ "learning_rate": 8.102564102564103e-05,
582
+ "loss": 0.2992,
583
+ "step": 74
584
+ },
585
+ {
586
+ "epoch": 1.896,
587
+ "grad_norm": 1.493260383605957,
588
+ "learning_rate": 8.076923076923078e-05,
589
+ "loss": 0.3058,
590
+ "step": 75
591
+ },
592
+ {
593
+ "epoch": 1.9216,
594
+ "grad_norm": 1.1610751152038574,
595
+ "learning_rate": 8.051282051282052e-05,
596
+ "loss": 0.2328,
597
+ "step": 76
598
+ },
599
+ {
600
+ "epoch": 1.9472,
601
+ "grad_norm": 1.2894799709320068,
602
+ "learning_rate": 8.025641025641026e-05,
603
+ "loss": 0.3086,
604
+ "step": 77
605
+ },
606
+ {
607
+ "epoch": 1.9727999999999999,
608
+ "grad_norm": 1.655768871307373,
609
+ "learning_rate": 8e-05,
610
+ "loss": 0.2247,
611
+ "step": 78
612
+ },
613
+ {
614
+ "epoch": 1.9984,
615
+ "grad_norm": 1.2705315351486206,
616
+ "learning_rate": 7.974358974358975e-05,
617
+ "loss": 0.2229,
618
+ "step": 79
619
+ },
620
+ {
621
+ "epoch": 2.0,
622
+ "grad_norm": 0.3245500326156616,
623
+ "learning_rate": 7.948717948717948e-05,
624
+ "loss": 0.0082,
625
+ "step": 80
626
+ },
627
+ {
628
+ "epoch": 2.0,
629
+ "eval_loss": 0.18327006697654724,
630
+ "eval_runtime": 46.2169,
631
+ "eval_samples_per_second": 9.758,
632
+ "eval_steps_per_second": 0.173,
633
+ "step": 80
634
+ },
635
+ {
636
+ "epoch": 2.0256,
637
+ "grad_norm": 1.2532856464385986,
638
+ "learning_rate": 7.923076923076924e-05,
639
+ "loss": 0.1692,
640
+ "step": 81
641
+ },
642
+ {
643
+ "epoch": 2.0512,
644
+ "grad_norm": 1.3299051523208618,
645
+ "learning_rate": 7.897435897435898e-05,
646
+ "loss": 0.2139,
647
+ "step": 82
648
+ },
649
+ {
650
+ "epoch": 2.0768,
651
+ "grad_norm": 1.3725135326385498,
652
+ "learning_rate": 7.871794871794872e-05,
653
+ "loss": 0.2156,
654
+ "step": 83
655
+ },
656
+ {
657
+ "epoch": 2.1024,
658
+ "grad_norm": 1.0960525274276733,
659
+ "learning_rate": 7.846153846153847e-05,
660
+ "loss": 0.2208,
661
+ "step": 84
662
+ },
663
+ {
664
+ "epoch": 2.128,
665
+ "grad_norm": 1.8501545190811157,
666
+ "learning_rate": 7.820512820512821e-05,
667
+ "loss": 0.1551,
668
+ "step": 85
669
+ },
670
+ {
671
+ "epoch": 2.1536,
672
+ "grad_norm": 1.0871448516845703,
673
+ "learning_rate": 7.794871794871795e-05,
674
+ "loss": 0.1282,
675
+ "step": 86
676
+ },
677
+ {
678
+ "epoch": 2.1792,
679
+ "grad_norm": 1.4489119052886963,
680
+ "learning_rate": 7.76923076923077e-05,
681
+ "loss": 0.2375,
682
+ "step": 87
683
+ },
684
+ {
685
+ "epoch": 2.2048,
686
+ "grad_norm": 1.5719850063323975,
687
+ "learning_rate": 7.743589743589744e-05,
688
+ "loss": 0.2503,
689
+ "step": 88
690
+ },
691
+ {
692
+ "epoch": 2.2304,
693
+ "grad_norm": 2.2236907482147217,
694
+ "learning_rate": 7.717948717948718e-05,
695
+ "loss": 0.2384,
696
+ "step": 89
697
+ },
698
+ {
699
+ "epoch": 2.2560000000000002,
700
+ "grad_norm": 1.2721396684646606,
701
+ "learning_rate": 7.692307692307693e-05,
702
+ "loss": 0.1752,
703
+ "step": 90
704
+ },
705
+ {
706
+ "epoch": 2.2560000000000002,
707
+ "eval_loss": 0.19305120408535004,
708
+ "eval_runtime": 46.1652,
709
+ "eval_samples_per_second": 9.769,
710
+ "eval_steps_per_second": 0.173,
711
+ "step": 90
712
+ },
713
+ {
714
+ "epoch": 2.2816,
715
+ "grad_norm": 0.9860581159591675,
716
+ "learning_rate": 7.666666666666667e-05,
717
+ "loss": 0.138,
718
+ "step": 91
719
+ },
720
+ {
721
+ "epoch": 2.3072,
722
+ "grad_norm": 1.7323637008666992,
723
+ "learning_rate": 7.641025641025641e-05,
724
+ "loss": 0.144,
725
+ "step": 92
726
+ },
727
+ {
728
+ "epoch": 2.3327999999999998,
729
+ "grad_norm": 1.7598427534103394,
730
+ "learning_rate": 7.615384615384616e-05,
731
+ "loss": 0.1451,
732
+ "step": 93
733
+ },
734
+ {
735
+ "epoch": 2.3584,
736
+ "grad_norm": 1.6885058879852295,
737
+ "learning_rate": 7.58974358974359e-05,
738
+ "loss": 0.1625,
739
+ "step": 94
740
+ },
741
+ {
742
+ "epoch": 2.384,
743
+ "grad_norm": 1.3391759395599365,
744
+ "learning_rate": 7.564102564102564e-05,
745
+ "loss": 0.2039,
746
+ "step": 95
747
+ },
748
+ {
749
+ "epoch": 2.4096,
750
+ "grad_norm": 3.5374937057495117,
751
+ "learning_rate": 7.538461538461539e-05,
752
+ "loss": 0.2374,
753
+ "step": 96
754
+ },
755
+ {
756
+ "epoch": 2.4352,
757
+ "grad_norm": 1.5867712497711182,
758
+ "learning_rate": 7.512820512820513e-05,
759
+ "loss": 0.1915,
760
+ "step": 97
761
+ },
762
+ {
763
+ "epoch": 2.4608,
764
+ "grad_norm": 1.6900838613510132,
765
+ "learning_rate": 7.487179487179487e-05,
766
+ "loss": 0.1946,
767
+ "step": 98
768
+ },
769
+ {
770
+ "epoch": 2.4864,
771
+ "grad_norm": 1.116487741470337,
772
+ "learning_rate": 7.461538461538462e-05,
773
+ "loss": 0.1056,
774
+ "step": 99
775
+ },
776
+ {
777
+ "epoch": 2.512,
778
+ "grad_norm": 1.9830695390701294,
779
+ "learning_rate": 7.435897435897436e-05,
780
+ "loss": 0.2071,
781
+ "step": 100
782
+ },
783
+ {
784
+ "epoch": 2.512,
785
+ "eval_loss": 0.18597476184368134,
786
+ "eval_runtime": 46.0769,
787
+ "eval_samples_per_second": 9.788,
788
+ "eval_steps_per_second": 0.174,
789
+ "step": 100
790
+ },
791
+ {
792
+ "epoch": 2.5376,
793
+ "grad_norm": 1.5822725296020508,
794
+ "learning_rate": 7.410256410256412e-05,
795
+ "loss": 0.1704,
796
+ "step": 101
797
+ },
798
+ {
799
+ "epoch": 2.5632,
800
+ "grad_norm": 2.2399463653564453,
801
+ "learning_rate": 7.384615384615386e-05,
802
+ "loss": 0.118,
803
+ "step": 102
804
+ },
805
+ {
806
+ "epoch": 2.5888,
807
+ "grad_norm": 1.2649646997451782,
808
+ "learning_rate": 7.35897435897436e-05,
809
+ "loss": 0.1192,
810
+ "step": 103
811
+ },
812
+ {
813
+ "epoch": 2.6144,
814
+ "grad_norm": 3.1921226978302,
815
+ "learning_rate": 7.333333333333333e-05,
816
+ "loss": 0.186,
817
+ "step": 104
818
+ },
819
+ {
820
+ "epoch": 2.64,
821
+ "grad_norm": 2.00970196723938,
822
+ "learning_rate": 7.307692307692307e-05,
823
+ "loss": 0.1235,
824
+ "step": 105
825
+ },
826
+ {
827
+ "epoch": 2.6656,
828
+ "grad_norm": 3.3171231746673584,
829
+ "learning_rate": 7.282051282051282e-05,
830
+ "loss": 0.164,
831
+ "step": 106
832
+ },
833
+ {
834
+ "epoch": 2.6912000000000003,
835
+ "grad_norm": 3.0452377796173096,
836
+ "learning_rate": 7.256410256410256e-05,
837
+ "loss": 0.2077,
838
+ "step": 107
839
+ },
840
+ {
841
+ "epoch": 2.7168,
842
+ "grad_norm": 1.9057981967926025,
843
+ "learning_rate": 7.23076923076923e-05,
844
+ "loss": 0.2009,
845
+ "step": 108
846
+ },
847
+ {
848
+ "epoch": 2.7424,
849
+ "grad_norm": 1.4594613313674927,
850
+ "learning_rate": 7.205128205128205e-05,
851
+ "loss": 0.1402,
852
+ "step": 109
853
+ },
854
+ {
855
+ "epoch": 2.768,
856
+ "grad_norm": 4.038388252258301,
857
+ "learning_rate": 7.17948717948718e-05,
858
+ "loss": 0.2379,
859
+ "step": 110
860
+ },
861
+ {
862
+ "epoch": 2.768,
863
+ "eval_loss": 0.1910053789615631,
864
+ "eval_runtime": 46.105,
865
+ "eval_samples_per_second": 9.782,
866
+ "eval_steps_per_second": 0.174,
867
+ "step": 110
868
+ },
869
+ {
870
+ "epoch": 2.7936,
871
+ "grad_norm": 1.4920284748077393,
872
+ "learning_rate": 7.153846153846155e-05,
873
+ "loss": 0.1924,
874
+ "step": 111
875
+ },
876
+ {
877
+ "epoch": 2.8192,
878
+ "grad_norm": 2.1072158813476562,
879
+ "learning_rate": 7.128205128205129e-05,
880
+ "loss": 0.2131,
881
+ "step": 112
882
+ },
883
+ {
884
+ "epoch": 2.8448,
885
+ "grad_norm": 2.7636947631835938,
886
+ "learning_rate": 7.102564102564103e-05,
887
+ "loss": 0.1955,
888
+ "step": 113
889
+ },
890
+ {
891
+ "epoch": 2.8704,
892
+ "grad_norm": 2.5713424682617188,
893
+ "learning_rate": 7.076923076923078e-05,
894
+ "loss": 0.2524,
895
+ "step": 114
896
+ },
897
+ {
898
+ "epoch": 2.896,
899
+ "grad_norm": 1.9095423221588135,
900
+ "learning_rate": 7.051282051282052e-05,
901
+ "loss": 0.1939,
902
+ "step": 115
903
+ },
904
+ {
905
+ "epoch": 2.9215999999999998,
906
+ "grad_norm": 2.3303332328796387,
907
+ "learning_rate": 7.025641025641025e-05,
908
+ "loss": 0.1829,
909
+ "step": 116
910
+ },
911
+ {
912
+ "epoch": 2.9472,
913
+ "grad_norm": 1.6265172958374023,
914
+ "learning_rate": 7e-05,
915
+ "loss": 0.1818,
916
+ "step": 117
917
+ },
918
+ {
919
+ "epoch": 2.9728,
920
+ "grad_norm": 1.802085518836975,
921
+ "learning_rate": 6.974358974358974e-05,
922
+ "loss": 0.1759,
923
+ "step": 118
924
+ },
925
+ {
926
+ "epoch": 2.9984,
927
+ "grad_norm": 2.0247466564178467,
928
+ "learning_rate": 6.94871794871795e-05,
929
+ "loss": 0.1263,
930
+ "step": 119
931
+ },
932
+ {
933
+ "epoch": 3.0,
934
+ "grad_norm": 0.0567646324634552,
935
+ "learning_rate": 6.923076923076924e-05,
936
+ "loss": 0.0012,
937
+ "step": 120
938
+ },
939
+ {
940
+ "epoch": 3.0,
941
+ "eval_loss": 0.1671789288520813,
942
+ "eval_runtime": 46.0792,
943
+ "eval_samples_per_second": 9.788,
944
+ "eval_steps_per_second": 0.174,
945
+ "step": 120
946
+ },
947
+ {
948
+ "epoch": 3.0256,
949
+ "grad_norm": 1.1815694570541382,
950
+ "learning_rate": 6.897435897435898e-05,
951
+ "loss": 0.0875,
952
+ "step": 121
953
+ },
954
+ {
955
+ "epoch": 3.0512,
956
+ "grad_norm": 2.159958839416504,
957
+ "learning_rate": 6.871794871794872e-05,
958
+ "loss": 0.2338,
959
+ "step": 122
960
+ },
961
+ {
962
+ "epoch": 3.0768,
963
+ "grad_norm": 0.8782948851585388,
964
+ "learning_rate": 6.846153846153847e-05,
965
+ "loss": 0.0931,
966
+ "step": 123
967
+ },
968
+ {
969
+ "epoch": 3.1024,
970
+ "grad_norm": 1.0242409706115723,
971
+ "learning_rate": 6.820512820512821e-05,
972
+ "loss": 0.089,
973
+ "step": 124
974
+ },
975
+ {
976
+ "epoch": 3.128,
977
+ "grad_norm": 1.9822003841400146,
978
+ "learning_rate": 6.794871794871795e-05,
979
+ "loss": 0.2043,
980
+ "step": 125
981
+ },
982
+ {
983
+ "epoch": 3.1536,
984
+ "grad_norm": 1.6593323945999146,
985
+ "learning_rate": 6.76923076923077e-05,
986
+ "loss": 0.0855,
987
+ "step": 126
988
+ },
989
+ {
990
+ "epoch": 3.1792,
991
+ "grad_norm": 1.2175663709640503,
992
+ "learning_rate": 6.743589743589744e-05,
993
+ "loss": 0.0522,
994
+ "step": 127
995
+ },
996
+ {
997
+ "epoch": 3.2048,
998
+ "grad_norm": 1.3775852918624878,
999
+ "learning_rate": 6.717948717948718e-05,
1000
+ "loss": 0.1653,
1001
+ "step": 128
1002
+ },
1003
+ {
1004
+ "epoch": 3.2304,
1005
+ "grad_norm": 1.052992820739746,
1006
+ "learning_rate": 6.692307692307693e-05,
1007
+ "loss": 0.0829,
1008
+ "step": 129
1009
+ },
1010
+ {
1011
+ "epoch": 3.2560000000000002,
1012
+ "grad_norm": 1.744211196899414,
1013
+ "learning_rate": 6.666666666666667e-05,
1014
+ "loss": 0.1709,
1015
+ "step": 130
1016
+ },
1017
+ {
1018
+ "epoch": 3.2560000000000002,
1019
+ "eval_loss": 0.1594507247209549,
1020
+ "eval_runtime": 46.0782,
1021
+ "eval_samples_per_second": 9.788,
1022
+ "eval_steps_per_second": 0.174,
1023
+ "step": 130
1024
+ },
1025
+ {
1026
+ "epoch": 3.2816,
1027
+ "grad_norm": 3.3828606605529785,
1028
+ "learning_rate": 6.641025641025641e-05,
1029
+ "loss": 0.1371,
1030
+ "step": 131
1031
+ },
1032
+ {
1033
+ "epoch": 3.3072,
1034
+ "grad_norm": 1.9074350595474243,
1035
+ "learning_rate": 6.615384615384616e-05,
1036
+ "loss": 0.1507,
1037
+ "step": 132
1038
+ },
1039
+ {
1040
+ "epoch": 3.3327999999999998,
1041
+ "grad_norm": 2.2225406169891357,
1042
+ "learning_rate": 6.58974358974359e-05,
1043
+ "loss": 0.1531,
1044
+ "step": 133
1045
+ },
1046
+ {
1047
+ "epoch": 3.3584,
1048
+ "grad_norm": 2.2324278354644775,
1049
+ "learning_rate": 6.564102564102564e-05,
1050
+ "loss": 0.1032,
1051
+ "step": 134
1052
+ },
1053
+ {
1054
+ "epoch": 3.384,
1055
+ "grad_norm": 1.0784149169921875,
1056
+ "learning_rate": 6.538461538461539e-05,
1057
+ "loss": 0.1118,
1058
+ "step": 135
1059
+ },
1060
+ {
1061
+ "epoch": 3.4096,
1062
+ "grad_norm": 1.7640796899795532,
1063
+ "learning_rate": 6.512820512820513e-05,
1064
+ "loss": 0.128,
1065
+ "step": 136
1066
+ },
1067
+ {
1068
+ "epoch": 3.4352,
1069
+ "grad_norm": 1.2802343368530273,
1070
+ "learning_rate": 6.487179487179487e-05,
1071
+ "loss": 0.1023,
1072
+ "step": 137
1073
+ },
1074
+ {
1075
+ "epoch": 3.4608,
1076
+ "grad_norm": 2.069298028945923,
1077
+ "learning_rate": 6.461538461538462e-05,
1078
+ "loss": 0.0983,
1079
+ "step": 138
1080
+ },
1081
+ {
1082
+ "epoch": 3.4864,
1083
+ "grad_norm": 1.988155722618103,
1084
+ "learning_rate": 6.435897435897437e-05,
1085
+ "loss": 0.1016,
1086
+ "step": 139
1087
+ },
1088
+ {
1089
+ "epoch": 3.512,
1090
+ "grad_norm": 2.035534620285034,
1091
+ "learning_rate": 6.410256410256412e-05,
1092
+ "loss": 0.1486,
1093
+ "step": 140
1094
+ },
1095
+ {
1096
+ "epoch": 3.512,
1097
+ "eval_loss": 0.15938659012317657,
1098
+ "eval_runtime": 46.1666,
1099
+ "eval_samples_per_second": 9.769,
1100
+ "eval_steps_per_second": 0.173,
1101
+ "step": 140
1102
+ },
1103
+ {
1104
+ "epoch": 3.5376,
1105
+ "grad_norm": 1.5117779970169067,
1106
+ "learning_rate": 6.384615384615385e-05,
1107
+ "loss": 0.1346,
1108
+ "step": 141
1109
+ },
1110
+ {
1111
+ "epoch": 3.5632,
1112
+ "grad_norm": 3.2372398376464844,
1113
+ "learning_rate": 6.358974358974359e-05,
1114
+ "loss": 0.0628,
1115
+ "step": 142
1116
+ },
1117
+ {
1118
+ "epoch": 3.5888,
1119
+ "grad_norm": 1.734154462814331,
1120
+ "learning_rate": 6.333333333333333e-05,
1121
+ "loss": 0.1006,
1122
+ "step": 143
1123
+ },
1124
+ {
1125
+ "epoch": 3.6144,
1126
+ "grad_norm": 1.8823364973068237,
1127
+ "learning_rate": 6.307692307692308e-05,
1128
+ "loss": 0.1361,
1129
+ "step": 144
1130
+ },
1131
+ {
1132
+ "epoch": 3.64,
1133
+ "grad_norm": 1.515404462814331,
1134
+ "learning_rate": 6.282051282051282e-05,
1135
+ "loss": 0.0794,
1136
+ "step": 145
1137
+ },
1138
+ {
1139
+ "epoch": 3.6656,
1140
+ "grad_norm": 1.3684951066970825,
1141
+ "learning_rate": 6.256410256410256e-05,
1142
+ "loss": 0.0917,
1143
+ "step": 146
1144
+ },
1145
+ {
1146
+ "epoch": 3.6912000000000003,
1147
+ "grad_norm": 1.0272871255874634,
1148
+ "learning_rate": 6.23076923076923e-05,
1149
+ "loss": 0.0695,
1150
+ "step": 147
1151
+ },
1152
+ {
1153
+ "epoch": 3.7168,
1154
+ "grad_norm": 2.0209293365478516,
1155
+ "learning_rate": 6.205128205128206e-05,
1156
+ "loss": 0.1134,
1157
+ "step": 148
1158
+ },
1159
+ {
1160
+ "epoch": 3.7424,
1161
+ "grad_norm": 1.2262115478515625,
1162
+ "learning_rate": 6.17948717948718e-05,
1163
+ "loss": 0.0909,
1164
+ "step": 149
1165
+ },
1166
+ {
1167
+ "epoch": 3.768,
1168
+ "grad_norm": 1.7901175022125244,
1169
+ "learning_rate": 6.153846153846155e-05,
1170
+ "loss": 0.0651,
1171
+ "step": 150
1172
+ },
1173
+ {
1174
+ "epoch": 3.768,
1175
+ "eval_loss": 0.16204459965229034,
1176
+ "eval_runtime": 46.1014,
1177
+ "eval_samples_per_second": 9.783,
1178
+ "eval_steps_per_second": 0.174,
1179
+ "step": 150
1180
+ },
1181
+ {
1182
+ "epoch": 3.7936,
1183
+ "grad_norm": 1.6963258981704712,
1184
+ "learning_rate": 6.128205128205129e-05,
1185
+ "loss": 0.1277,
1186
+ "step": 151
1187
+ },
1188
+ {
1189
+ "epoch": 3.8192,
1190
+ "grad_norm": 3.0077695846557617,
1191
+ "learning_rate": 6.1025641025641035e-05,
1192
+ "loss": 0.1935,
1193
+ "step": 152
1194
+ },
1195
+ {
1196
+ "epoch": 3.8448,
1197
+ "grad_norm": 2.1329452991485596,
1198
+ "learning_rate": 6.0769230769230765e-05,
1199
+ "loss": 0.1412,
1200
+ "step": 153
1201
+ },
1202
+ {
1203
+ "epoch": 3.8704,
1204
+ "grad_norm": 3.3125712871551514,
1205
+ "learning_rate": 6.0512820512820515e-05,
1206
+ "loss": 0.18,
1207
+ "step": 154
1208
+ },
1209
+ {
1210
+ "epoch": 3.896,
1211
+ "grad_norm": 1.4981236457824707,
1212
+ "learning_rate": 6.025641025641026e-05,
1213
+ "loss": 0.0833,
1214
+ "step": 155
1215
+ },
1216
+ {
1217
+ "epoch": 3.9215999999999998,
1218
+ "grad_norm": 1.3236380815505981,
1219
+ "learning_rate": 6e-05,
1220
+ "loss": 0.1045,
1221
+ "step": 156
1222
+ },
1223
+ {
1224
+ "epoch": 3.9472,
1225
+ "grad_norm": 2.1060631275177,
1226
+ "learning_rate": 5.9743589743589745e-05,
1227
+ "loss": 0.0907,
1228
+ "step": 157
1229
+ },
1230
+ {
1231
+ "epoch": 3.9728,
1232
+ "grad_norm": 0.9833385348320007,
1233
+ "learning_rate": 5.948717948717949e-05,
1234
+ "loss": 0.0502,
1235
+ "step": 158
1236
+ },
1237
+ {
1238
+ "epoch": 3.9984,
1239
+ "grad_norm": 1.5143266916275024,
1240
+ "learning_rate": 5.923076923076923e-05,
1241
+ "loss": 0.0822,
1242
+ "step": 159
1243
+ },
1244
+ {
1245
+ "epoch": 4.0,
1246
+ "grad_norm": 0.31218358874320984,
1247
+ "learning_rate": 5.897435897435898e-05,
1248
+ "loss": 0.0043,
1249
+ "step": 160
1250
+ },
1251
+ {
1252
+ "epoch": 4.0,
1253
+ "eval_loss": 0.1569308638572693,
1254
+ "eval_runtime": 46.0425,
1255
+ "eval_samples_per_second": 9.795,
1256
+ "eval_steps_per_second": 0.174,
1257
+ "step": 160
1258
+ },
1259
+ {
1260
+ "epoch": 4.0256,
1261
+ "grad_norm": 1.2191623449325562,
1262
+ "learning_rate": 5.8717948717948725e-05,
1263
+ "loss": 0.0555,
1264
+ "step": 161
1265
+ },
1266
+ {
1267
+ "epoch": 4.0512,
1268
+ "grad_norm": 1.833415150642395,
1269
+ "learning_rate": 5.846153846153847e-05,
1270
+ "loss": 0.069,
1271
+ "step": 162
1272
+ },
1273
+ {
1274
+ "epoch": 4.0768,
1275
+ "grad_norm": 2.106884717941284,
1276
+ "learning_rate": 5.820512820512821e-05,
1277
+ "loss": 0.1464,
1278
+ "step": 163
1279
+ },
1280
+ {
1281
+ "epoch": 4.1024,
1282
+ "grad_norm": 1.7515654563903809,
1283
+ "learning_rate": 5.7948717948717954e-05,
1284
+ "loss": 0.0675,
1285
+ "step": 164
1286
+ },
1287
+ {
1288
+ "epoch": 4.128,
1289
+ "grad_norm": 1.447368860244751,
1290
+ "learning_rate": 5.769230769230769e-05,
1291
+ "loss": 0.0677,
1292
+ "step": 165
1293
+ },
1294
+ {
1295
+ "epoch": 4.1536,
1296
+ "grad_norm": 2.6953341960906982,
1297
+ "learning_rate": 5.7435897435897434e-05,
1298
+ "loss": 0.099,
1299
+ "step": 166
1300
+ },
1301
+ {
1302
+ "epoch": 4.1792,
1303
+ "grad_norm": 1.5634942054748535,
1304
+ "learning_rate": 5.717948717948718e-05,
1305
+ "loss": 0.0596,
1306
+ "step": 167
1307
+ },
1308
+ {
1309
+ "epoch": 4.2048,
1310
+ "grad_norm": 1.276308298110962,
1311
+ "learning_rate": 5.692307692307692e-05,
1312
+ "loss": 0.0661,
1313
+ "step": 168
1314
+ },
1315
+ {
1316
+ "epoch": 4.2304,
1317
+ "grad_norm": 1.228543996810913,
1318
+ "learning_rate": 5.666666666666667e-05,
1319
+ "loss": 0.0808,
1320
+ "step": 169
1321
+ },
1322
+ {
1323
+ "epoch": 4.256,
1324
+ "grad_norm": 1.6373244524002075,
1325
+ "learning_rate": 5.6410256410256414e-05,
1326
+ "loss": 0.0465,
1327
+ "step": 170
1328
+ },
1329
+ {
1330
+ "epoch": 4.256,
1331
+ "eval_loss": 0.15826600790023804,
1332
+ "eval_runtime": 46.1301,
1333
+ "eval_samples_per_second": 9.777,
1334
+ "eval_steps_per_second": 0.173,
1335
+ "step": 170
1336
+ },
1337
+ {
1338
+ "epoch": 4.2816,
1339
+ "grad_norm": 1.3432047367095947,
1340
+ "learning_rate": 5.615384615384616e-05,
1341
+ "loss": 0.0755,
1342
+ "step": 171
1343
+ },
1344
+ {
1345
+ "epoch": 4.3072,
1346
+ "grad_norm": 1.2605931758880615,
1347
+ "learning_rate": 5.58974358974359e-05,
1348
+ "loss": 0.0611,
1349
+ "step": 172
1350
+ },
1351
+ {
1352
+ "epoch": 4.3328,
1353
+ "grad_norm": 2.181140422821045,
1354
+ "learning_rate": 5.5641025641025644e-05,
1355
+ "loss": 0.0615,
1356
+ "step": 173
1357
+ },
1358
+ {
1359
+ "epoch": 4.3584,
1360
+ "grad_norm": 1.6840100288391113,
1361
+ "learning_rate": 5.538461538461539e-05,
1362
+ "loss": 0.0814,
1363
+ "step": 174
1364
+ },
1365
+ {
1366
+ "epoch": 4.384,
1367
+ "grad_norm": 1.543228268623352,
1368
+ "learning_rate": 5.512820512820514e-05,
1369
+ "loss": 0.1456,
1370
+ "step": 175
1371
+ },
1372
+ {
1373
+ "epoch": 4.4096,
1374
+ "grad_norm": 1.5495244264602661,
1375
+ "learning_rate": 5.487179487179488e-05,
1376
+ "loss": 0.07,
1377
+ "step": 176
1378
+ },
1379
+ {
1380
+ "epoch": 4.4352,
1381
+ "grad_norm": 2.249027729034424,
1382
+ "learning_rate": 5.461538461538461e-05,
1383
+ "loss": 0.0488,
1384
+ "step": 177
1385
+ },
1386
+ {
1387
+ "epoch": 4.4608,
1388
+ "grad_norm": 3.975053310394287,
1389
+ "learning_rate": 5.435897435897436e-05,
1390
+ "loss": 0.1022,
1391
+ "step": 178
1392
+ },
1393
+ {
1394
+ "epoch": 4.4864,
1395
+ "grad_norm": 1.0509039163589478,
1396
+ "learning_rate": 5.41025641025641e-05,
1397
+ "loss": 0.0395,
1398
+ "step": 179
1399
+ },
1400
+ {
1401
+ "epoch": 4.5120000000000005,
1402
+ "grad_norm": 0.7361845374107361,
1403
+ "learning_rate": 5.384615384615385e-05,
1404
+ "loss": 0.0288,
1405
+ "step": 180
1406
+ },
1407
+ {
1408
+ "epoch": 4.5120000000000005,
1409
+ "eval_loss": 0.15827181935310364,
1410
+ "eval_runtime": 46.0746,
1411
+ "eval_samples_per_second": 9.788,
1412
+ "eval_steps_per_second": 0.174,
1413
+ "step": 180
1414
+ },
1415
+ {
1416
+ "epoch": 4.5376,
1417
+ "grad_norm": 1.3167545795440674,
1418
+ "learning_rate": 5.358974358974359e-05,
1419
+ "loss": 0.0413,
1420
+ "step": 181
1421
+ },
1422
+ {
1423
+ "epoch": 4.5632,
1424
+ "grad_norm": 1.1776018142700195,
1425
+ "learning_rate": 5.333333333333333e-05,
1426
+ "loss": 0.0289,
1427
+ "step": 182
1428
+ },
1429
+ {
1430
+ "epoch": 4.5888,
1431
+ "grad_norm": 1.1779769659042358,
1432
+ "learning_rate": 5.3076923076923076e-05,
1433
+ "loss": 0.0451,
1434
+ "step": 183
1435
+ },
1436
+ {
1437
+ "epoch": 4.6144,
1438
+ "grad_norm": 1.191728949546814,
1439
+ "learning_rate": 5.2820512820512826e-05,
1440
+ "loss": 0.0402,
1441
+ "step": 184
1442
+ },
1443
+ {
1444
+ "epoch": 4.64,
1445
+ "grad_norm": 1.8995720148086548,
1446
+ "learning_rate": 5.256410256410257e-05,
1447
+ "loss": 0.0407,
1448
+ "step": 185
1449
+ },
1450
+ {
1451
+ "epoch": 4.6655999999999995,
1452
+ "grad_norm": 2.597722053527832,
1453
+ "learning_rate": 5.230769230769231e-05,
1454
+ "loss": 0.059,
1455
+ "step": 186
1456
+ },
1457
+ {
1458
+ "epoch": 4.6912,
1459
+ "grad_norm": 2.1279001235961914,
1460
+ "learning_rate": 5.2051282051282056e-05,
1461
+ "loss": 0.07,
1462
+ "step": 187
1463
+ },
1464
+ {
1465
+ "epoch": 4.7168,
1466
+ "grad_norm": 2.0430350303649902,
1467
+ "learning_rate": 5.17948717948718e-05,
1468
+ "loss": 0.0555,
1469
+ "step": 188
1470
+ },
1471
+ {
1472
+ "epoch": 4.7424,
1473
+ "grad_norm": 2.5439860820770264,
1474
+ "learning_rate": 5.1538461538461536e-05,
1475
+ "loss": 0.0994,
1476
+ "step": 189
1477
+ },
1478
+ {
1479
+ "epoch": 4.768,
1480
+ "grad_norm": 2.0388362407684326,
1481
+ "learning_rate": 5.128205128205128e-05,
1482
+ "loss": 0.0665,
1483
+ "step": 190
1484
+ },
1485
+ {
1486
+ "epoch": 4.768,
1487
+ "eval_loss": 0.15733125805854797,
1488
+ "eval_runtime": 46.0902,
1489
+ "eval_samples_per_second": 9.785,
1490
+ "eval_steps_per_second": 0.174,
1491
+ "step": 190
1492
+ },
1493
+ {
1494
+ "epoch": 4.7936,
1495
+ "grad_norm": 2.7209362983703613,
1496
+ "learning_rate": 5.102564102564102e-05,
1497
+ "loss": 0.0489,
1498
+ "step": 191
1499
+ },
1500
+ {
1501
+ "epoch": 4.8192,
1502
+ "grad_norm": 1.3868178129196167,
1503
+ "learning_rate": 5.0769230769230766e-05,
1504
+ "loss": 0.0666,
1505
+ "step": 192
1506
+ },
1507
+ {
1508
+ "epoch": 4.8448,
1509
+ "grad_norm": 0.9440222978591919,
1510
+ "learning_rate": 5.0512820512820516e-05,
1511
+ "loss": 0.0259,
1512
+ "step": 193
1513
+ },
1514
+ {
1515
+ "epoch": 4.8704,
1516
+ "grad_norm": 1.782159447669983,
1517
+ "learning_rate": 5.025641025641026e-05,
1518
+ "loss": 0.0532,
1519
+ "step": 194
1520
+ },
1521
+ {
1522
+ "epoch": 4.896,
1523
+ "grad_norm": 2.1891701221466064,
1524
+ "learning_rate": 5e-05,
1525
+ "loss": 0.0369,
1526
+ "step": 195
1527
+ },
1528
+ {
1529
+ "epoch": 4.9216,
1530
+ "grad_norm": 0.9858984351158142,
1531
+ "learning_rate": 4.9743589743589746e-05,
1532
+ "loss": 0.0248,
1533
+ "step": 196
1534
+ },
1535
+ {
1536
+ "epoch": 4.9472000000000005,
1537
+ "grad_norm": 1.814439296722412,
1538
+ "learning_rate": 4.948717948717949e-05,
1539
+ "loss": 0.0494,
1540
+ "step": 197
1541
+ },
1542
+ {
1543
+ "epoch": 4.9728,
1544
+ "grad_norm": 2.066479206085205,
1545
+ "learning_rate": 4.923076923076924e-05,
1546
+ "loss": 0.0626,
1547
+ "step": 198
1548
+ },
1549
+ {
1550
+ "epoch": 4.9984,
1551
+ "grad_norm": 0.9697967767715454,
1552
+ "learning_rate": 4.8974358974358975e-05,
1553
+ "loss": 0.0416,
1554
+ "step": 199
1555
+ },
1556
+ {
1557
+ "epoch": 5.0,
1558
+ "grad_norm": 0.023776618763804436,
1559
+ "learning_rate": 4.871794871794872e-05,
1560
+ "loss": 0.0002,
1561
+ "step": 200
1562
+ },
1563
+ {
1564
+ "epoch": 5.0,
1565
+ "eval_loss": 0.1582992523908615,
1566
+ "eval_runtime": 46.0662,
1567
+ "eval_samples_per_second": 9.79,
1568
+ "eval_steps_per_second": 0.174,
1569
+ "step": 200
1570
+ },
1571
+ {
1572
+ "epoch": 5.0256,
1573
+ "grad_norm": 1.4514086246490479,
1574
+ "learning_rate": 4.846153846153846e-05,
1575
+ "loss": 0.0326,
1576
+ "step": 201
1577
+ },
1578
+ {
1579
+ "epoch": 5.0512,
1580
+ "grad_norm": 1.0607144832611084,
1581
+ "learning_rate": 4.8205128205128205e-05,
1582
+ "loss": 0.0347,
1583
+ "step": 202
1584
+ },
1585
+ {
1586
+ "epoch": 5.0768,
1587
+ "grad_norm": 1.9190572500228882,
1588
+ "learning_rate": 4.7948717948717955e-05,
1589
+ "loss": 0.028,
1590
+ "step": 203
1591
+ },
1592
+ {
1593
+ "epoch": 5.1024,
1594
+ "grad_norm": 1.476803183555603,
1595
+ "learning_rate": 4.76923076923077e-05,
1596
+ "loss": 0.0287,
1597
+ "step": 204
1598
+ },
1599
+ {
1600
+ "epoch": 5.128,
1601
+ "grad_norm": 1.7281403541564941,
1602
+ "learning_rate": 4.7435897435897435e-05,
1603
+ "loss": 0.0422,
1604
+ "step": 205
1605
+ },
1606
+ {
1607
+ "epoch": 5.1536,
1608
+ "grad_norm": 1.5540275573730469,
1609
+ "learning_rate": 4.717948717948718e-05,
1610
+ "loss": 0.0219,
1611
+ "step": 206
1612
+ },
1613
+ {
1614
+ "epoch": 5.1792,
1615
+ "grad_norm": 0.6488132476806641,
1616
+ "learning_rate": 4.692307692307693e-05,
1617
+ "loss": 0.0177,
1618
+ "step": 207
1619
+ },
1620
+ {
1621
+ "epoch": 5.2048,
1622
+ "grad_norm": 0.6170593500137329,
1623
+ "learning_rate": 4.666666666666667e-05,
1624
+ "loss": 0.0156,
1625
+ "step": 208
1626
+ },
1627
+ {
1628
+ "epoch": 5.2304,
1629
+ "grad_norm": 1.2647796869277954,
1630
+ "learning_rate": 4.6410256410256415e-05,
1631
+ "loss": 0.0204,
1632
+ "step": 209
1633
+ },
1634
+ {
1635
+ "epoch": 5.256,
1636
+ "grad_norm": 2.475916624069214,
1637
+ "learning_rate": 4.615384615384616e-05,
1638
+ "loss": 0.0306,
1639
+ "step": 210
1640
+ },
1641
+ {
1642
+ "epoch": 5.256,
1643
+ "eval_loss": 0.16460593044757843,
1644
+ "eval_runtime": 46.1754,
1645
+ "eval_samples_per_second": 9.767,
1646
+ "eval_steps_per_second": 0.173,
1647
+ "step": 210
1648
+ },
1649
+ {
1650
+ "epoch": 5.2816,
1651
+ "grad_norm": 2.594714879989624,
1652
+ "learning_rate": 4.5897435897435895e-05,
1653
+ "loss": 0.0271,
1654
+ "step": 211
1655
+ },
1656
+ {
1657
+ "epoch": 5.3072,
1658
+ "grad_norm": 1.516627550125122,
1659
+ "learning_rate": 4.5641025641025645e-05,
1660
+ "loss": 0.0443,
1661
+ "step": 212
1662
+ },
1663
+ {
1664
+ "epoch": 5.3328,
1665
+ "grad_norm": 0.8540337681770325,
1666
+ "learning_rate": 4.538461538461539e-05,
1667
+ "loss": 0.018,
1668
+ "step": 213
1669
+ },
1670
+ {
1671
+ "epoch": 5.3584,
1672
+ "grad_norm": 0.8031948208808899,
1673
+ "learning_rate": 4.512820512820513e-05,
1674
+ "loss": 0.0144,
1675
+ "step": 214
1676
+ },
1677
+ {
1678
+ "epoch": 5.384,
1679
+ "grad_norm": 1.550010085105896,
1680
+ "learning_rate": 4.4871794871794874e-05,
1681
+ "loss": 0.0256,
1682
+ "step": 215
1683
+ },
1684
+ {
1685
+ "epoch": 5.4096,
1686
+ "grad_norm": 0.9779123067855835,
1687
+ "learning_rate": 4.461538461538462e-05,
1688
+ "loss": 0.0217,
1689
+ "step": 216
1690
+ },
1691
+ {
1692
+ "epoch": 5.4352,
1693
+ "grad_norm": 1.3814575672149658,
1694
+ "learning_rate": 4.435897435897436e-05,
1695
+ "loss": 0.0509,
1696
+ "step": 217
1697
+ },
1698
+ {
1699
+ "epoch": 5.4608,
1700
+ "grad_norm": 0.6444527506828308,
1701
+ "learning_rate": 4.4102564102564104e-05,
1702
+ "loss": 0.0128,
1703
+ "step": 218
1704
+ },
1705
+ {
1706
+ "epoch": 5.4864,
1707
+ "grad_norm": 2.447659730911255,
1708
+ "learning_rate": 4.384615384615385e-05,
1709
+ "loss": 0.0408,
1710
+ "step": 219
1711
+ },
1712
+ {
1713
+ "epoch": 5.5120000000000005,
1714
+ "grad_norm": 1.4684360027313232,
1715
+ "learning_rate": 4.358974358974359e-05,
1716
+ "loss": 0.0282,
1717
+ "step": 220
1718
+ },
1719
+ {
1720
+ "epoch": 5.5120000000000005,
1721
+ "eval_loss": 0.17198146879673004,
1722
+ "eval_runtime": 46.1144,
1723
+ "eval_samples_per_second": 9.78,
1724
+ "eval_steps_per_second": 0.173,
1725
+ "step": 220
1726
+ },
1727
+ {
1728
+ "epoch": 5.5376,
1729
+ "grad_norm": 2.4431331157684326,
1730
+ "learning_rate": 4.3333333333333334e-05,
1731
+ "loss": 0.0577,
1732
+ "step": 221
1733
+ },
1734
+ {
1735
+ "epoch": 5.5632,
1736
+ "grad_norm": 2.405123472213745,
1737
+ "learning_rate": 4.3076923076923084e-05,
1738
+ "loss": 0.0601,
1739
+ "step": 222
1740
+ },
1741
+ {
1742
+ "epoch": 5.5888,
1743
+ "grad_norm": 1.342482089996338,
1744
+ "learning_rate": 4.282051282051282e-05,
1745
+ "loss": 0.0211,
1746
+ "step": 223
1747
+ },
1748
+ {
1749
+ "epoch": 5.6144,
1750
+ "grad_norm": 0.6261419057846069,
1751
+ "learning_rate": 4.2564102564102564e-05,
1752
+ "loss": 0.0104,
1753
+ "step": 224
1754
+ },
1755
+ {
1756
+ "epoch": 5.64,
1757
+ "grad_norm": 1.443263053894043,
1758
+ "learning_rate": 4.230769230769231e-05,
1759
+ "loss": 0.0306,
1760
+ "step": 225
1761
+ },
1762
+ {
1763
+ "epoch": 5.6655999999999995,
1764
+ "grad_norm": 3.3630077838897705,
1765
+ "learning_rate": 4.205128205128206e-05,
1766
+ "loss": 0.0346,
1767
+ "step": 226
1768
+ },
1769
+ {
1770
+ "epoch": 5.6912,
1771
+ "grad_norm": 1.3521815538406372,
1772
+ "learning_rate": 4.17948717948718e-05,
1773
+ "loss": 0.0186,
1774
+ "step": 227
1775
+ },
1776
+ {
1777
+ "epoch": 5.7168,
1778
+ "grad_norm": 0.9824311137199402,
1779
+ "learning_rate": 4.1538461538461544e-05,
1780
+ "loss": 0.0094,
1781
+ "step": 228
1782
+ },
1783
+ {
1784
+ "epoch": 5.7424,
1785
+ "grad_norm": 0.9648368954658508,
1786
+ "learning_rate": 4.128205128205128e-05,
1787
+ "loss": 0.025,
1788
+ "step": 229
1789
+ },
1790
+ {
1791
+ "epoch": 5.768,
1792
+ "grad_norm": 1.2381614446640015,
1793
+ "learning_rate": 4.1025641025641023e-05,
1794
+ "loss": 0.0225,
1795
+ "step": 230
1796
+ },
1797
+ {
1798
+ "epoch": 5.768,
1799
+ "eval_loss": 0.16620177030563354,
1800
+ "eval_runtime": 46.0894,
1801
+ "eval_samples_per_second": 9.785,
1802
+ "eval_steps_per_second": 0.174,
1803
+ "step": 230
1804
+ },
1805
+ {
1806
+ "epoch": 5.7936,
1807
+ "grad_norm": 1.3803844451904297,
1808
+ "learning_rate": 4.0769230769230773e-05,
1809
+ "loss": 0.0277,
1810
+ "step": 231
1811
+ },
1812
+ {
1813
+ "epoch": 5.8192,
1814
+ "grad_norm": 2.083890914916992,
1815
+ "learning_rate": 4.051282051282052e-05,
1816
+ "loss": 0.0315,
1817
+ "step": 232
1818
+ },
1819
+ {
1820
+ "epoch": 5.8448,
1821
+ "grad_norm": 2.827819347381592,
1822
+ "learning_rate": 4.025641025641026e-05,
1823
+ "loss": 0.0809,
1824
+ "step": 233
1825
+ },
1826
+ {
1827
+ "epoch": 5.8704,
1828
+ "grad_norm": 1.4013893604278564,
1829
+ "learning_rate": 4e-05,
1830
+ "loss": 0.0731,
1831
+ "step": 234
1832
+ },
1833
+ {
1834
+ "epoch": 5.896,
1835
+ "grad_norm": 1.5475901365280151,
1836
+ "learning_rate": 3.974358974358974e-05,
1837
+ "loss": 0.0309,
1838
+ "step": 235
1839
+ },
1840
+ {
1841
+ "epoch": 5.9216,
1842
+ "grad_norm": 2.6997406482696533,
1843
+ "learning_rate": 3.948717948717949e-05,
1844
+ "loss": 0.061,
1845
+ "step": 236
1846
+ },
1847
+ {
1848
+ "epoch": 5.9472000000000005,
1849
+ "grad_norm": 0.8604904413223267,
1850
+ "learning_rate": 3.923076923076923e-05,
1851
+ "loss": 0.0142,
1852
+ "step": 237
1853
+ },
1854
+ {
1855
+ "epoch": 5.9728,
1856
+ "grad_norm": 1.4761265516281128,
1857
+ "learning_rate": 3.8974358974358976e-05,
1858
+ "loss": 0.0182,
1859
+ "step": 238
1860
+ },
1861
+ {
1862
+ "epoch": 5.9984,
1863
+ "grad_norm": 1.3573832511901855,
1864
+ "learning_rate": 3.871794871794872e-05,
1865
+ "loss": 0.0186,
1866
+ "step": 239
1867
+ },
1868
+ {
1869
+ "epoch": 6.0,
1870
+ "grad_norm": 0.05791207030415535,
1871
+ "learning_rate": 3.846153846153846e-05,
1872
+ "loss": 0.0004,
1873
+ "step": 240
1874
+ },
1875
+ {
1876
+ "epoch": 6.0,
1877
+ "eval_loss": 0.1659858524799347,
1878
+ "eval_runtime": 46.0508,
1879
+ "eval_samples_per_second": 9.794,
1880
+ "eval_steps_per_second": 0.174,
1881
+ "step": 240
1882
+ },
1883
+ {
1884
+ "epoch": 6.0256,
1885
+ "grad_norm": 0.8057020902633667,
1886
+ "learning_rate": 3.8205128205128206e-05,
1887
+ "loss": 0.0148,
1888
+ "step": 241
1889
+ },
1890
+ {
1891
+ "epoch": 6.0512,
1892
+ "grad_norm": 1.3963549137115479,
1893
+ "learning_rate": 3.794871794871795e-05,
1894
+ "loss": 0.0311,
1895
+ "step": 242
1896
+ },
1897
+ {
1898
+ "epoch": 6.0768,
1899
+ "grad_norm": 0.4920665919780731,
1900
+ "learning_rate": 3.769230769230769e-05,
1901
+ "loss": 0.007,
1902
+ "step": 243
1903
+ },
1904
+ {
1905
+ "epoch": 6.1024,
1906
+ "grad_norm": 0.5270194411277771,
1907
+ "learning_rate": 3.7435897435897436e-05,
1908
+ "loss": 0.0112,
1909
+ "step": 244
1910
+ },
1911
+ {
1912
+ "epoch": 6.128,
1913
+ "grad_norm": 1.5079636573791504,
1914
+ "learning_rate": 3.717948717948718e-05,
1915
+ "loss": 0.033,
1916
+ "step": 245
1917
+ },
1918
+ {
1919
+ "epoch": 6.1536,
1920
+ "grad_norm": 0.7430498600006104,
1921
+ "learning_rate": 3.692307692307693e-05,
1922
+ "loss": 0.0088,
1923
+ "step": 246
1924
+ },
1925
+ {
1926
+ "epoch": 6.1792,
1927
+ "grad_norm": 1.5670939683914185,
1928
+ "learning_rate": 3.6666666666666666e-05,
1929
+ "loss": 0.0111,
1930
+ "step": 247
1931
+ },
1932
+ {
1933
+ "epoch": 6.2048,
1934
+ "grad_norm": 1.0994242429733276,
1935
+ "learning_rate": 3.641025641025641e-05,
1936
+ "loss": 0.0284,
1937
+ "step": 248
1938
+ },
1939
+ {
1940
+ "epoch": 6.2304,
1941
+ "grad_norm": 1.008639931678772,
1942
+ "learning_rate": 3.615384615384615e-05,
1943
+ "loss": 0.0112,
1944
+ "step": 249
1945
+ },
1946
+ {
1947
+ "epoch": 6.256,
1948
+ "grad_norm": 0.6727498769760132,
1949
+ "learning_rate": 3.58974358974359e-05,
1950
+ "loss": 0.01,
1951
+ "step": 250
1952
+ },
1953
+ {
1954
+ "epoch": 6.256,
1955
+ "eval_loss": 0.16884523630142212,
1956
+ "eval_runtime": 46.0532,
1957
+ "eval_samples_per_second": 9.793,
1958
+ "eval_steps_per_second": 0.174,
1959
+ "step": 250
1960
+ },
1961
+ {
1962
+ "epoch": 6.2816,
1963
+ "grad_norm": 0.7288702726364136,
1964
+ "learning_rate": 3.5641025641025646e-05,
1965
+ "loss": 0.0095,
1966
+ "step": 251
1967
+ },
1968
+ {
1969
+ "epoch": 6.3072,
1970
+ "grad_norm": 0.5252525806427002,
1971
+ "learning_rate": 3.538461538461539e-05,
1972
+ "loss": 0.0064,
1973
+ "step": 252
1974
+ },
1975
+ {
1976
+ "epoch": 6.3328,
1977
+ "grad_norm": 0.30171850323677063,
1978
+ "learning_rate": 3.5128205128205125e-05,
1979
+ "loss": 0.0046,
1980
+ "step": 253
1981
+ },
1982
+ {
1983
+ "epoch": 6.3584,
1984
+ "grad_norm": 0.8617231845855713,
1985
+ "learning_rate": 3.487179487179487e-05,
1986
+ "loss": 0.0103,
1987
+ "step": 254
1988
+ },
1989
+ {
1990
+ "epoch": 6.384,
1991
+ "grad_norm": 0.37915322184562683,
1992
+ "learning_rate": 3.461538461538462e-05,
1993
+ "loss": 0.006,
1994
+ "step": 255
1995
+ },
1996
+ {
1997
+ "epoch": 6.4096,
1998
+ "grad_norm": 0.2540806829929352,
1999
+ "learning_rate": 3.435897435897436e-05,
2000
+ "loss": 0.0044,
2001
+ "step": 256
2002
+ },
2003
+ {
2004
+ "epoch": 6.4352,
2005
+ "grad_norm": 1.2370727062225342,
2006
+ "learning_rate": 3.4102564102564105e-05,
2007
+ "loss": 0.0643,
2008
+ "step": 257
2009
+ },
2010
+ {
2011
+ "epoch": 6.4608,
2012
+ "grad_norm": 1.992367148399353,
2013
+ "learning_rate": 3.384615384615385e-05,
2014
+ "loss": 0.0203,
2015
+ "step": 258
2016
+ },
2017
+ {
2018
+ "epoch": 6.4864,
2019
+ "grad_norm": 0.20070694386959076,
2020
+ "learning_rate": 3.358974358974359e-05,
2021
+ "loss": 0.003,
2022
+ "step": 259
2023
+ },
2024
+ {
2025
+ "epoch": 6.5120000000000005,
2026
+ "grad_norm": 1.786927342414856,
2027
+ "learning_rate": 3.3333333333333335e-05,
2028
+ "loss": 0.0301,
2029
+ "step": 260
2030
+ },
2031
+ {
2032
+ "epoch": 6.5120000000000005,
2033
+ "eval_loss": 0.18007855117321014,
2034
+ "eval_runtime": 46.2331,
2035
+ "eval_samples_per_second": 9.755,
2036
+ "eval_steps_per_second": 0.173,
2037
+ "step": 260
2038
+ },
2039
+ {
2040
+ "epoch": 6.5376,
2041
+ "grad_norm": 0.2549300789833069,
2042
+ "learning_rate": 3.307692307692308e-05,
2043
+ "loss": 0.0042,
2044
+ "step": 261
2045
+ },
2046
+ {
2047
+ "epoch": 6.5632,
2048
+ "grad_norm": 0.47428637742996216,
2049
+ "learning_rate": 3.282051282051282e-05,
2050
+ "loss": 0.0033,
2051
+ "step": 262
2052
+ },
2053
+ {
2054
+ "epoch": 6.5888,
2055
+ "grad_norm": 0.4179193079471588,
2056
+ "learning_rate": 3.2564102564102565e-05,
2057
+ "loss": 0.0056,
2058
+ "step": 263
2059
+ },
2060
+ {
2061
+ "epoch": 6.6144,
2062
+ "grad_norm": 0.7487953901290894,
2063
+ "learning_rate": 3.230769230769231e-05,
2064
+ "loss": 0.0053,
2065
+ "step": 264
2066
+ },
2067
+ {
2068
+ "epoch": 6.64,
2069
+ "grad_norm": 1.241440773010254,
2070
+ "learning_rate": 3.205128205128206e-05,
2071
+ "loss": 0.0092,
2072
+ "step": 265
2073
+ },
2074
+ {
2075
+ "epoch": 6.6655999999999995,
2076
+ "grad_norm": 1.0357407331466675,
2077
+ "learning_rate": 3.1794871794871795e-05,
2078
+ "loss": 0.0127,
2079
+ "step": 266
2080
+ },
2081
+ {
2082
+ "epoch": 6.6912,
2083
+ "grad_norm": 1.5997498035430908,
2084
+ "learning_rate": 3.153846153846154e-05,
2085
+ "loss": 0.0146,
2086
+ "step": 267
2087
+ },
2088
+ {
2089
+ "epoch": 6.7168,
2090
+ "grad_norm": 0.2820545732975006,
2091
+ "learning_rate": 3.128205128205128e-05,
2092
+ "loss": 0.0032,
2093
+ "step": 268
2094
+ },
2095
+ {
2096
+ "epoch": 6.7424,
2097
+ "grad_norm": 1.378670334815979,
2098
+ "learning_rate": 3.102564102564103e-05,
2099
+ "loss": 0.0073,
2100
+ "step": 269
2101
+ },
2102
+ {
2103
+ "epoch": 6.768,
2104
+ "grad_norm": 0.5203121304512024,
2105
+ "learning_rate": 3.0769230769230774e-05,
2106
+ "loss": 0.0089,
2107
+ "step": 270
2108
+ },
2109
+ {
2110
+ "epoch": 6.768,
2111
+ "eval_loss": 0.18411996960639954,
2112
+ "eval_runtime": 46.0604,
2113
+ "eval_samples_per_second": 9.791,
2114
+ "eval_steps_per_second": 0.174,
2115
+ "step": 270
2116
+ },
2117
+ {
2118
+ "epoch": 6.7936,
2119
+ "grad_norm": 0.44411182403564453,
2120
+ "learning_rate": 3.0512820512820518e-05,
2121
+ "loss": 0.003,
2122
+ "step": 271
2123
+ },
2124
+ {
2125
+ "epoch": 6.8192,
2126
+ "grad_norm": 1.3348493576049805,
2127
+ "learning_rate": 3.0256410256410257e-05,
2128
+ "loss": 0.0166,
2129
+ "step": 272
2130
+ },
2131
+ {
2132
+ "epoch": 6.8448,
2133
+ "grad_norm": 0.26802995800971985,
2134
+ "learning_rate": 3e-05,
2135
+ "loss": 0.0043,
2136
+ "step": 273
2137
+ },
2138
+ {
2139
+ "epoch": 6.8704,
2140
+ "grad_norm": 0.3201054036617279,
2141
+ "learning_rate": 2.9743589743589744e-05,
2142
+ "loss": 0.003,
2143
+ "step": 274
2144
+ },
2145
+ {
2146
+ "epoch": 6.896,
2147
+ "grad_norm": 1.0242953300476074,
2148
+ "learning_rate": 2.948717948717949e-05,
2149
+ "loss": 0.0116,
2150
+ "step": 275
2151
+ },
2152
+ {
2153
+ "epoch": 6.9216,
2154
+ "grad_norm": 0.33335596323013306,
2155
+ "learning_rate": 2.9230769230769234e-05,
2156
+ "loss": 0.0043,
2157
+ "step": 276
2158
+ },
2159
+ {
2160
+ "epoch": 6.9472000000000005,
2161
+ "grad_norm": 1.5013763904571533,
2162
+ "learning_rate": 2.8974358974358977e-05,
2163
+ "loss": 0.0179,
2164
+ "step": 277
2165
+ },
2166
+ {
2167
+ "epoch": 6.9728,
2168
+ "grad_norm": 1.779761791229248,
2169
+ "learning_rate": 2.8717948717948717e-05,
2170
+ "loss": 0.0303,
2171
+ "step": 278
2172
+ },
2173
+ {
2174
+ "epoch": 6.9984,
2175
+ "grad_norm": 0.758307933807373,
2176
+ "learning_rate": 2.846153846153846e-05,
2177
+ "loss": 0.0136,
2178
+ "step": 279
2179
+ },
2180
+ {
2181
+ "epoch": 7.0,
2182
+ "grad_norm": 1.0144366025924683,
2183
+ "learning_rate": 2.8205128205128207e-05,
2184
+ "loss": 0.0079,
2185
+ "step": 280
2186
+ },
2187
+ {
2188
+ "epoch": 7.0,
2189
+ "eval_loss": 0.19086241722106934,
2190
+ "eval_runtime": 46.0659,
2191
+ "eval_samples_per_second": 9.79,
2192
+ "eval_steps_per_second": 0.174,
2193
+ "step": 280
2194
+ },
2195
+ {
2196
+ "epoch": 7.0256,
2197
+ "grad_norm": 0.37199220061302185,
2198
+ "learning_rate": 2.794871794871795e-05,
2199
+ "loss": 0.0041,
2200
+ "step": 281
2201
+ },
2202
+ {
2203
+ "epoch": 7.0512,
2204
+ "grad_norm": 0.7925446629524231,
2205
+ "learning_rate": 2.7692307692307694e-05,
2206
+ "loss": 0.0055,
2207
+ "step": 282
2208
+ },
2209
+ {
2210
+ "epoch": 7.0768,
2211
+ "grad_norm": 0.2660503387451172,
2212
+ "learning_rate": 2.743589743589744e-05,
2213
+ "loss": 0.0028,
2214
+ "step": 283
2215
+ },
2216
+ {
2217
+ "epoch": 7.1024,
2218
+ "grad_norm": 0.9306926727294922,
2219
+ "learning_rate": 2.717948717948718e-05,
2220
+ "loss": 0.0465,
2221
+ "step": 284
2222
+ },
2223
+ {
2224
+ "epoch": 7.128,
2225
+ "grad_norm": 0.6875828504562378,
2226
+ "learning_rate": 2.6923076923076923e-05,
2227
+ "loss": 0.0091,
2228
+ "step": 285
2229
+ },
2230
+ {
2231
+ "epoch": 7.1536,
2232
+ "grad_norm": 0.16312462091445923,
2233
+ "learning_rate": 2.6666666666666667e-05,
2234
+ "loss": 0.0023,
2235
+ "step": 286
2236
+ },
2237
+ {
2238
+ "epoch": 7.1792,
2239
+ "grad_norm": 0.6696649789810181,
2240
+ "learning_rate": 2.6410256410256413e-05,
2241
+ "loss": 0.0089,
2242
+ "step": 287
2243
+ },
2244
+ {
2245
+ "epoch": 7.2048,
2246
+ "grad_norm": 0.3207850754261017,
2247
+ "learning_rate": 2.6153846153846157e-05,
2248
+ "loss": 0.0039,
2249
+ "step": 288
2250
+ },
2251
+ {
2252
+ "epoch": 7.2304,
2253
+ "grad_norm": 0.34680280089378357,
2254
+ "learning_rate": 2.58974358974359e-05,
2255
+ "loss": 0.0033,
2256
+ "step": 289
2257
+ },
2258
+ {
2259
+ "epoch": 7.256,
2260
+ "grad_norm": 0.19657476246356964,
2261
+ "learning_rate": 2.564102564102564e-05,
2262
+ "loss": 0.003,
2263
+ "step": 290
2264
+ },
2265
+ {
2266
+ "epoch": 7.256,
2267
+ "eval_loss": 0.19000744819641113,
2268
+ "eval_runtime": 46.0565,
2269
+ "eval_samples_per_second": 9.792,
2270
+ "eval_steps_per_second": 0.174,
2271
+ "step": 290
2272
+ },
2273
+ {
2274
+ "epoch": 7.2816,
2275
+ "grad_norm": 0.2564392685890198,
2276
+ "learning_rate": 2.5384615384615383e-05,
2277
+ "loss": 0.004,
2278
+ "step": 291
2279
+ },
2280
+ {
2281
+ "epoch": 7.3072,
2282
+ "grad_norm": 0.782609224319458,
2283
+ "learning_rate": 2.512820512820513e-05,
2284
+ "loss": 0.0083,
2285
+ "step": 292
2286
+ },
2287
+ {
2288
+ "epoch": 7.3328,
2289
+ "grad_norm": 0.1740560382604599,
2290
+ "learning_rate": 2.4871794871794873e-05,
2291
+ "loss": 0.0022,
2292
+ "step": 293
2293
+ },
2294
+ {
2295
+ "epoch": 7.3584,
2296
+ "grad_norm": 0.08479921519756317,
2297
+ "learning_rate": 2.461538461538462e-05,
2298
+ "loss": 0.0016,
2299
+ "step": 294
2300
+ },
2301
+ {
2302
+ "epoch": 7.384,
2303
+ "grad_norm": 0.38202646374702454,
2304
+ "learning_rate": 2.435897435897436e-05,
2305
+ "loss": 0.0042,
2306
+ "step": 295
2307
+ },
2308
+ {
2309
+ "epoch": 7.4096,
2310
+ "grad_norm": 0.20064568519592285,
2311
+ "learning_rate": 2.4102564102564103e-05,
2312
+ "loss": 0.002,
2313
+ "step": 296
2314
+ },
2315
+ {
2316
+ "epoch": 7.4352,
2317
+ "grad_norm": 0.2550853490829468,
2318
+ "learning_rate": 2.384615384615385e-05,
2319
+ "loss": 0.0026,
2320
+ "step": 297
2321
+ },
2322
+ {
2323
+ "epoch": 7.4608,
2324
+ "grad_norm": 0.48428618907928467,
2325
+ "learning_rate": 2.358974358974359e-05,
2326
+ "loss": 0.0054,
2327
+ "step": 298
2328
+ },
2329
+ {
2330
+ "epoch": 7.4864,
2331
+ "grad_norm": 1.1436755657196045,
2332
+ "learning_rate": 2.3333333333333336e-05,
2333
+ "loss": 0.0152,
2334
+ "step": 299
2335
+ },
2336
+ {
2337
+ "epoch": 7.5120000000000005,
2338
+ "grad_norm": 0.2809373140335083,
2339
+ "learning_rate": 2.307692307692308e-05,
2340
+ "loss": 0.0018,
2341
+ "step": 300
2342
+ },
2343
+ {
2344
+ "epoch": 7.5120000000000005,
2345
+ "eval_loss": 0.19469289481639862,
2346
+ "eval_runtime": 46.0372,
2347
+ "eval_samples_per_second": 9.796,
2348
+ "eval_steps_per_second": 0.174,
2349
+ "step": 300
2350
+ },
2351
+ {
2352
+ "epoch": 7.5376,
2353
+ "grad_norm": 1.0407063961029053,
2354
+ "learning_rate": 2.2820512820512822e-05,
2355
+ "loss": 0.0124,
2356
+ "step": 301
2357
+ },
2358
+ {
2359
+ "epoch": 7.5632,
2360
+ "grad_norm": 0.3781401515007019,
2361
+ "learning_rate": 2.2564102564102566e-05,
2362
+ "loss": 0.0024,
2363
+ "step": 302
2364
+ },
2365
+ {
2366
+ "epoch": 7.5888,
2367
+ "grad_norm": 0.3046801686286926,
2368
+ "learning_rate": 2.230769230769231e-05,
2369
+ "loss": 0.0028,
2370
+ "step": 303
2371
+ },
2372
+ {
2373
+ "epoch": 7.6144,
2374
+ "grad_norm": 0.17133867740631104,
2375
+ "learning_rate": 2.2051282051282052e-05,
2376
+ "loss": 0.002,
2377
+ "step": 304
2378
+ },
2379
+ {
2380
+ "epoch": 7.64,
2381
+ "grad_norm": 0.29253873229026794,
2382
+ "learning_rate": 2.1794871794871795e-05,
2383
+ "loss": 0.0049,
2384
+ "step": 305
2385
+ },
2386
+ {
2387
+ "epoch": 7.6655999999999995,
2388
+ "grad_norm": 0.6607497930526733,
2389
+ "learning_rate": 2.1538461538461542e-05,
2390
+ "loss": 0.0152,
2391
+ "step": 306
2392
+ },
2393
+ {
2394
+ "epoch": 7.6912,
2395
+ "grad_norm": 0.4728367328643799,
2396
+ "learning_rate": 2.1282051282051282e-05,
2397
+ "loss": 0.0049,
2398
+ "step": 307
2399
+ },
2400
+ {
2401
+ "epoch": 7.7168,
2402
+ "grad_norm": 0.5030187368392944,
2403
+ "learning_rate": 2.102564102564103e-05,
2404
+ "loss": 0.0037,
2405
+ "step": 308
2406
+ },
2407
+ {
2408
+ "epoch": 7.7424,
2409
+ "grad_norm": 0.11288933455944061,
2410
+ "learning_rate": 2.0769230769230772e-05,
2411
+ "loss": 0.0012,
2412
+ "step": 309
2413
+ },
2414
+ {
2415
+ "epoch": 7.768,
2416
+ "grad_norm": 0.4386049211025238,
2417
+ "learning_rate": 2.0512820512820512e-05,
2418
+ "loss": 0.0059,
2419
+ "step": 310
2420
+ },
2421
+ {
2422
+ "epoch": 7.768,
2423
+ "eval_loss": 0.19445903599262238,
2424
+ "eval_runtime": 46.0284,
2425
+ "eval_samples_per_second": 9.798,
2426
+ "eval_steps_per_second": 0.174,
2427
+ "step": 310
2428
+ },
2429
+ {
2430
+ "epoch": 7.7936,
2431
+ "grad_norm": 0.15119102597236633,
2432
+ "learning_rate": 2.025641025641026e-05,
2433
+ "loss": 0.0016,
2434
+ "step": 311
2435
+ },
2436
+ {
2437
+ "epoch": 7.8192,
2438
+ "grad_norm": 0.36368149518966675,
2439
+ "learning_rate": 2e-05,
2440
+ "loss": 0.0037,
2441
+ "step": 312
2442
+ },
2443
+ {
2444
+ "epoch": 7.8448,
2445
+ "grad_norm": 0.22931738197803497,
2446
+ "learning_rate": 1.9743589743589745e-05,
2447
+ "loss": 0.0023,
2448
+ "step": 313
2449
+ },
2450
+ {
2451
+ "epoch": 7.8704,
2452
+ "grad_norm": 0.30464252829551697,
2453
+ "learning_rate": 1.9487179487179488e-05,
2454
+ "loss": 0.0027,
2455
+ "step": 314
2456
+ },
2457
+ {
2458
+ "epoch": 7.896,
2459
+ "grad_norm": 0.13730935752391815,
2460
+ "learning_rate": 1.923076923076923e-05,
2461
+ "loss": 0.0017,
2462
+ "step": 315
2463
+ },
2464
+ {
2465
+ "epoch": 7.9216,
2466
+ "grad_norm": 0.21119461953639984,
2467
+ "learning_rate": 1.8974358974358975e-05,
2468
+ "loss": 0.0031,
2469
+ "step": 316
2470
+ },
2471
+ {
2472
+ "epoch": 7.9472000000000005,
2473
+ "grad_norm": 1.3350460529327393,
2474
+ "learning_rate": 1.8717948717948718e-05,
2475
+ "loss": 0.0154,
2476
+ "step": 317
2477
+ },
2478
+ {
2479
+ "epoch": 7.9728,
2480
+ "grad_norm": 0.29415613412857056,
2481
+ "learning_rate": 1.8461538461538465e-05,
2482
+ "loss": 0.0025,
2483
+ "step": 318
2484
+ },
2485
+ {
2486
+ "epoch": 7.9984,
2487
+ "grad_norm": 0.20422235131263733,
2488
+ "learning_rate": 1.8205128205128204e-05,
2489
+ "loss": 0.0023,
2490
+ "step": 319
2491
+ },
2492
+ {
2493
+ "epoch": 8.0,
2494
+ "grad_norm": 0.01667635701596737,
2495
+ "learning_rate": 1.794871794871795e-05,
2496
+ "loss": 0.0001,
2497
+ "step": 320
2498
+ },
2499
+ {
2500
+ "epoch": 8.0,
2501
+ "eval_loss": 0.19959121942520142,
2502
+ "eval_runtime": 46.0703,
2503
+ "eval_samples_per_second": 9.789,
2504
+ "eval_steps_per_second": 0.174,
2505
+ "step": 320
2506
+ },
2507
+ {
2508
+ "epoch": 8.0256,
2509
+ "grad_norm": 0.3130502700805664,
2510
+ "learning_rate": 1.7692307692307694e-05,
2511
+ "loss": 0.003,
2512
+ "step": 321
2513
+ },
2514
+ {
2515
+ "epoch": 8.0512,
2516
+ "grad_norm": 0.5019899010658264,
2517
+ "learning_rate": 1.7435897435897434e-05,
2518
+ "loss": 0.0096,
2519
+ "step": 322
2520
+ },
2521
+ {
2522
+ "epoch": 8.0768,
2523
+ "grad_norm": 0.07362326979637146,
2524
+ "learning_rate": 1.717948717948718e-05,
2525
+ "loss": 0.0011,
2526
+ "step": 323
2527
+ },
2528
+ {
2529
+ "epoch": 8.1024,
2530
+ "grad_norm": 0.15674403309822083,
2531
+ "learning_rate": 1.6923076923076924e-05,
2532
+ "loss": 0.0018,
2533
+ "step": 324
2534
+ },
2535
+ {
2536
+ "epoch": 8.128,
2537
+ "grad_norm": 0.16153913736343384,
2538
+ "learning_rate": 1.6666666666666667e-05,
2539
+ "loss": 0.002,
2540
+ "step": 325
2541
+ },
2542
+ {
2543
+ "epoch": 8.1536,
2544
+ "grad_norm": 0.10829649865627289,
2545
+ "learning_rate": 1.641025641025641e-05,
2546
+ "loss": 0.001,
2547
+ "step": 326
2548
+ },
2549
+ {
2550
+ "epoch": 8.1792,
2551
+ "grad_norm": 0.13399221003055573,
2552
+ "learning_rate": 1.6153846153846154e-05,
2553
+ "loss": 0.0016,
2554
+ "step": 327
2555
+ },
2556
+ {
2557
+ "epoch": 8.2048,
2558
+ "grad_norm": 0.6621348857879639,
2559
+ "learning_rate": 1.5897435897435897e-05,
2560
+ "loss": 0.0061,
2561
+ "step": 328
2562
+ },
2563
+ {
2564
+ "epoch": 8.2304,
2565
+ "grad_norm": 0.12478233873844147,
2566
+ "learning_rate": 1.564102564102564e-05,
2567
+ "loss": 0.0013,
2568
+ "step": 329
2569
+ },
2570
+ {
2571
+ "epoch": 8.256,
2572
+ "grad_norm": 0.2321283519268036,
2573
+ "learning_rate": 1.5384615384615387e-05,
2574
+ "loss": 0.0024,
2575
+ "step": 330
2576
+ },
2577
+ {
2578
+ "epoch": 8.256,
2579
+ "eval_loss": 0.20678183436393738,
2580
+ "eval_runtime": 46.102,
2581
+ "eval_samples_per_second": 9.783,
2582
+ "eval_steps_per_second": 0.174,
2583
+ "step": 330
2584
+ },
2585
+ {
2586
+ "epoch": 8.2816,
2587
+ "grad_norm": 0.3146536648273468,
2588
+ "learning_rate": 1.5128205128205129e-05,
2589
+ "loss": 0.0028,
2590
+ "step": 331
2591
+ },
2592
+ {
2593
+ "epoch": 8.3072,
2594
+ "grad_norm": 0.34905433654785156,
2595
+ "learning_rate": 1.4871794871794872e-05,
2596
+ "loss": 0.0023,
2597
+ "step": 332
2598
+ },
2599
+ {
2600
+ "epoch": 8.3328,
2601
+ "grad_norm": 0.677470326423645,
2602
+ "learning_rate": 1.4615384615384617e-05,
2603
+ "loss": 0.0049,
2604
+ "step": 333
2605
+ },
2606
+ {
2607
+ "epoch": 8.3584,
2608
+ "grad_norm": 0.14551085233688354,
2609
+ "learning_rate": 1.4358974358974359e-05,
2610
+ "loss": 0.0014,
2611
+ "step": 334
2612
+ },
2613
+ {
2614
+ "epoch": 8.384,
2615
+ "grad_norm": 0.11376652866601944,
2616
+ "learning_rate": 1.4102564102564104e-05,
2617
+ "loss": 0.0015,
2618
+ "step": 335
2619
+ },
2620
+ {
2621
+ "epoch": 8.4096,
2622
+ "grad_norm": 0.6229234337806702,
2623
+ "learning_rate": 1.3846153846153847e-05,
2624
+ "loss": 0.0042,
2625
+ "step": 336
2626
+ },
2627
+ {
2628
+ "epoch": 8.4352,
2629
+ "grad_norm": 0.1884259432554245,
2630
+ "learning_rate": 1.358974358974359e-05,
2631
+ "loss": 0.0019,
2632
+ "step": 337
2633
+ },
2634
+ {
2635
+ "epoch": 8.4608,
2636
+ "grad_norm": 0.27811136841773987,
2637
+ "learning_rate": 1.3333333333333333e-05,
2638
+ "loss": 0.0023,
2639
+ "step": 338
2640
+ },
2641
+ {
2642
+ "epoch": 8.4864,
2643
+ "grad_norm": 0.07908125221729279,
2644
+ "learning_rate": 1.3076923076923078e-05,
2645
+ "loss": 0.0011,
2646
+ "step": 339
2647
+ },
2648
+ {
2649
+ "epoch": 8.512,
2650
+ "grad_norm": 0.15802721679210663,
2651
+ "learning_rate": 1.282051282051282e-05,
2652
+ "loss": 0.001,
2653
+ "step": 340
2654
+ },
2655
+ {
2656
+ "epoch": 8.512,
2657
+ "eval_loss": 0.20775224268436432,
2658
+ "eval_runtime": 46.0873,
2659
+ "eval_samples_per_second": 9.786,
2660
+ "eval_steps_per_second": 0.174,
2661
+ "step": 340
2662
+ },
2663
+ {
2664
+ "epoch": 8.5376,
2665
+ "grad_norm": 0.4371594488620758,
2666
+ "learning_rate": 1.2564102564102565e-05,
2667
+ "loss": 0.0039,
2668
+ "step": 341
2669
+ },
2670
+ {
2671
+ "epoch": 8.5632,
2672
+ "grad_norm": 0.7350974082946777,
2673
+ "learning_rate": 1.230769230769231e-05,
2674
+ "loss": 0.0058,
2675
+ "step": 342
2676
+ },
2677
+ {
2678
+ "epoch": 8.588799999999999,
2679
+ "grad_norm": 0.2187982201576233,
2680
+ "learning_rate": 1.2051282051282051e-05,
2681
+ "loss": 0.0019,
2682
+ "step": 343
2683
+ },
2684
+ {
2685
+ "epoch": 8.6144,
2686
+ "grad_norm": 0.36454564332962036,
2687
+ "learning_rate": 1.1794871794871795e-05,
2688
+ "loss": 0.0037,
2689
+ "step": 344
2690
+ },
2691
+ {
2692
+ "epoch": 8.64,
2693
+ "grad_norm": 0.2089586853981018,
2694
+ "learning_rate": 1.153846153846154e-05,
2695
+ "loss": 0.002,
2696
+ "step": 345
2697
+ },
2698
+ {
2699
+ "epoch": 8.6656,
2700
+ "grad_norm": 0.16004692018032074,
2701
+ "learning_rate": 1.1282051282051283e-05,
2702
+ "loss": 0.0016,
2703
+ "step": 346
2704
+ },
2705
+ {
2706
+ "epoch": 8.6912,
2707
+ "grad_norm": 0.13019374012947083,
2708
+ "learning_rate": 1.1025641025641026e-05,
2709
+ "loss": 0.0017,
2710
+ "step": 347
2711
+ },
2712
+ {
2713
+ "epoch": 8.7168,
2714
+ "grad_norm": 0.20604482293128967,
2715
+ "learning_rate": 1.0769230769230771e-05,
2716
+ "loss": 0.0028,
2717
+ "step": 348
2718
+ },
2719
+ {
2720
+ "epoch": 8.7424,
2721
+ "grad_norm": 0.2585042417049408,
2722
+ "learning_rate": 1.0512820512820514e-05,
2723
+ "loss": 0.0019,
2724
+ "step": 349
2725
+ },
2726
+ {
2727
+ "epoch": 8.768,
2728
+ "grad_norm": 0.13119027018547058,
2729
+ "learning_rate": 1.0256410256410256e-05,
2730
+ "loss": 0.0015,
2731
+ "step": 350
2732
+ },
2733
+ {
2734
+ "epoch": 8.768,
2735
+ "eval_loss": 0.2104375809431076,
2736
+ "eval_runtime": 46.0673,
2737
+ "eval_samples_per_second": 9.79,
2738
+ "eval_steps_per_second": 0.174,
2739
+ "step": 350
2740
+ },
2741
+ {
2742
+ "epoch": 8.7936,
2743
+ "grad_norm": 0.23227037489414215,
2744
+ "learning_rate": 1e-05,
2745
+ "loss": 0.0023,
2746
+ "step": 351
2747
+ },
2748
+ {
2749
+ "epoch": 8.8192,
2750
+ "grad_norm": 0.25416502356529236,
2751
+ "learning_rate": 9.743589743589744e-06,
2752
+ "loss": 0.0021,
2753
+ "step": 352
2754
+ },
2755
+ {
2756
+ "epoch": 8.8448,
2757
+ "grad_norm": 0.08733928948640823,
2758
+ "learning_rate": 9.487179487179487e-06,
2759
+ "loss": 0.0012,
2760
+ "step": 353
2761
+ },
2762
+ {
2763
+ "epoch": 8.8704,
2764
+ "grad_norm": 0.33069783449172974,
2765
+ "learning_rate": 9.230769230769232e-06,
2766
+ "loss": 0.0041,
2767
+ "step": 354
2768
+ },
2769
+ {
2770
+ "epoch": 8.896,
2771
+ "grad_norm": 1.0839825868606567,
2772
+ "learning_rate": 8.974358974358976e-06,
2773
+ "loss": 0.0438,
2774
+ "step": 355
2775
+ },
2776
+ {
2777
+ "epoch": 8.9216,
2778
+ "grad_norm": 0.23376193642616272,
2779
+ "learning_rate": 8.717948717948717e-06,
2780
+ "loss": 0.0028,
2781
+ "step": 356
2782
+ },
2783
+ {
2784
+ "epoch": 8.9472,
2785
+ "grad_norm": 0.2836299240589142,
2786
+ "learning_rate": 8.461538461538462e-06,
2787
+ "loss": 0.002,
2788
+ "step": 357
2789
+ },
2790
+ {
2791
+ "epoch": 8.9728,
2792
+ "grad_norm": 0.04910902678966522,
2793
+ "learning_rate": 8.205128205128205e-06,
2794
+ "loss": 0.0007,
2795
+ "step": 358
2796
+ },
2797
+ {
2798
+ "epoch": 8.9984,
2799
+ "grad_norm": 0.08654181659221649,
2800
+ "learning_rate": 7.948717948717949e-06,
2801
+ "loss": 0.0009,
2802
+ "step": 359
2803
+ },
2804
+ {
2805
+ "epoch": 9.0,
2806
+ "grad_norm": 0.016530824825167656,
2807
+ "learning_rate": 7.692307692307694e-06,
2808
+ "loss": 0.0001,
2809
+ "step": 360
2810
+ },
2811
+ {
2812
+ "epoch": 9.0,
2813
+ "eval_loss": 0.2104598730802536,
2814
+ "eval_runtime": 46.2138,
2815
+ "eval_samples_per_second": 9.759,
2816
+ "eval_steps_per_second": 0.173,
2817
+ "step": 360
2818
+ },
2819
+ {
2820
+ "epoch": 9.0256,
2821
+ "grad_norm": 0.19786006212234497,
2822
+ "learning_rate": 7.435897435897436e-06,
2823
+ "loss": 0.0029,
2824
+ "step": 361
2825
+ },
2826
+ {
2827
+ "epoch": 9.0512,
2828
+ "grad_norm": 0.15577854216098785,
2829
+ "learning_rate": 7.179487179487179e-06,
2830
+ "loss": 0.0012,
2831
+ "step": 362
2832
+ },
2833
+ {
2834
+ "epoch": 9.0768,
2835
+ "grad_norm": 0.09487804770469666,
2836
+ "learning_rate": 6.923076923076923e-06,
2837
+ "loss": 0.0014,
2838
+ "step": 363
2839
+ },
2840
+ {
2841
+ "epoch": 9.1024,
2842
+ "grad_norm": 0.1183762401342392,
2843
+ "learning_rate": 6.666666666666667e-06,
2844
+ "loss": 0.0013,
2845
+ "step": 364
2846
+ },
2847
+ {
2848
+ "epoch": 9.128,
2849
+ "grad_norm": 0.2357182800769806,
2850
+ "learning_rate": 6.41025641025641e-06,
2851
+ "loss": 0.0022,
2852
+ "step": 365
2853
+ },
2854
+ {
2855
+ "epoch": 9.1536,
2856
+ "grad_norm": 0.16520433127880096,
2857
+ "learning_rate": 6.153846153846155e-06,
2858
+ "loss": 0.0016,
2859
+ "step": 366
2860
+ },
2861
+ {
2862
+ "epoch": 9.1792,
2863
+ "grad_norm": 0.35106149315834045,
2864
+ "learning_rate": 5.897435897435897e-06,
2865
+ "loss": 0.0041,
2866
+ "step": 367
2867
+ },
2868
+ {
2869
+ "epoch": 9.2048,
2870
+ "grad_norm": 0.28743863105773926,
2871
+ "learning_rate": 5.641025641025641e-06,
2872
+ "loss": 0.0015,
2873
+ "step": 368
2874
+ },
2875
+ {
2876
+ "epoch": 9.2304,
2877
+ "grad_norm": 0.07648279517889023,
2878
+ "learning_rate": 5.3846153846153855e-06,
2879
+ "loss": 0.0009,
2880
+ "step": 369
2881
+ },
2882
+ {
2883
+ "epoch": 9.256,
2884
+ "grad_norm": 0.25945615768432617,
2885
+ "learning_rate": 5.128205128205128e-06,
2886
+ "loss": 0.0021,
2887
+ "step": 370
2888
+ },
2889
+ {
2890
+ "epoch": 9.256,
2891
+ "eval_loss": 0.2159082591533661,
2892
+ "eval_runtime": 46.0968,
2893
+ "eval_samples_per_second": 9.784,
2894
+ "eval_steps_per_second": 0.174,
2895
+ "step": 370
2896
+ },
2897
+ {
2898
+ "epoch": 9.2816,
2899
+ "grad_norm": 0.12486568838357925,
2900
+ "learning_rate": 4.871794871794872e-06,
2901
+ "loss": 0.0013,
2902
+ "step": 371
2903
+ },
2904
+ {
2905
+ "epoch": 9.3072,
2906
+ "grad_norm": 0.22302450239658356,
2907
+ "learning_rate": 4.615384615384616e-06,
2908
+ "loss": 0.0027,
2909
+ "step": 372
2910
+ },
2911
+ {
2912
+ "epoch": 9.3328,
2913
+ "grad_norm": 0.9940178990364075,
2914
+ "learning_rate": 4.3589743589743586e-06,
2915
+ "loss": 0.0376,
2916
+ "step": 373
2917
+ },
2918
+ {
2919
+ "epoch": 9.3584,
2920
+ "grad_norm": 0.0722355991601944,
2921
+ "learning_rate": 4.102564102564103e-06,
2922
+ "loss": 0.0009,
2923
+ "step": 374
2924
+ },
2925
+ {
2926
+ "epoch": 9.384,
2927
+ "grad_norm": 0.16168925166130066,
2928
+ "learning_rate": 3.846153846153847e-06,
2929
+ "loss": 0.0016,
2930
+ "step": 375
2931
+ },
2932
+ {
2933
+ "epoch": 9.4096,
2934
+ "grad_norm": 0.14245012402534485,
2935
+ "learning_rate": 3.5897435897435896e-06,
2936
+ "loss": 0.0012,
2937
+ "step": 376
2938
+ },
2939
+ {
2940
+ "epoch": 9.4352,
2941
+ "grad_norm": 0.6443990468978882,
2942
+ "learning_rate": 3.3333333333333333e-06,
2943
+ "loss": 0.0051,
2944
+ "step": 377
2945
+ },
2946
+ {
2947
+ "epoch": 9.4608,
2948
+ "grad_norm": 0.06897876411676407,
2949
+ "learning_rate": 3.0769230769230774e-06,
2950
+ "loss": 0.0008,
2951
+ "step": 378
2952
+ },
2953
+ {
2954
+ "epoch": 9.4864,
2955
+ "grad_norm": 0.06450845301151276,
2956
+ "learning_rate": 2.8205128205128207e-06,
2957
+ "loss": 0.0007,
2958
+ "step": 379
2959
+ },
2960
+ {
2961
+ "epoch": 9.512,
2962
+ "grad_norm": 0.16875888407230377,
2963
+ "learning_rate": 2.564102564102564e-06,
2964
+ "loss": 0.0017,
2965
+ "step": 380
2966
+ },
2967
+ {
2968
+ "epoch": 9.512,
2969
+ "eval_loss": 0.21007762849330902,
2970
+ "eval_runtime": 46.1576,
2971
+ "eval_samples_per_second": 9.771,
2972
+ "eval_steps_per_second": 0.173,
2973
+ "step": 380
2974
+ },
2975
+ {
2976
+ "epoch": 9.5376,
2977
+ "grad_norm": 0.08018496632575989,
2978
+ "learning_rate": 2.307692307692308e-06,
2979
+ "loss": 0.0008,
2980
+ "step": 381
2981
+ },
2982
+ {
2983
+ "epoch": 9.5632,
2984
+ "grad_norm": 0.22642257809638977,
2985
+ "learning_rate": 2.0512820512820513e-06,
2986
+ "loss": 0.0015,
2987
+ "step": 382
2988
+ },
2989
+ {
2990
+ "epoch": 9.588799999999999,
2991
+ "grad_norm": 0.5728319883346558,
2992
+ "learning_rate": 1.7948717948717948e-06,
2993
+ "loss": 0.0079,
2994
+ "step": 383
2995
+ },
2996
+ {
2997
+ "epoch": 9.6144,
2998
+ "grad_norm": 0.16731196641921997,
2999
+ "learning_rate": 1.5384615384615387e-06,
3000
+ "loss": 0.0012,
3001
+ "step": 384
3002
+ },
3003
+ {
3004
+ "epoch": 9.64,
3005
+ "grad_norm": 0.08967146277427673,
3006
+ "learning_rate": 1.282051282051282e-06,
3007
+ "loss": 0.0009,
3008
+ "step": 385
3009
+ },
3010
+ {
3011
+ "epoch": 9.6656,
3012
+ "grad_norm": 0.19866441190242767,
3013
+ "learning_rate": 1.0256410256410257e-06,
3014
+ "loss": 0.0023,
3015
+ "step": 386
3016
+ },
3017
+ {
3018
+ "epoch": 9.6912,
3019
+ "grad_norm": 0.07427145540714264,
3020
+ "learning_rate": 7.692307692307694e-07,
3021
+ "loss": 0.0009,
3022
+ "step": 387
3023
+ },
3024
+ {
3025
+ "epoch": 9.7168,
3026
+ "grad_norm": 0.2308674156665802,
3027
+ "learning_rate": 5.128205128205128e-07,
3028
+ "loss": 0.0022,
3029
+ "step": 388
3030
+ },
3031
+ {
3032
+ "epoch": 9.7424,
3033
+ "grad_norm": 0.4221293330192566,
3034
+ "learning_rate": 2.564102564102564e-07,
3035
+ "loss": 0.0047,
3036
+ "step": 389
3037
+ },
3038
+ {
3039
+ "epoch": 9.768,
3040
+ "grad_norm": 0.14237630367279053,
3041
+ "learning_rate": 0.0,
3042
+ "loss": 0.0014,
3043
+ "step": 390
3044
+ },
3045
+ {
3046
+ "epoch": 9.768,
3047
+ "eval_loss": 0.21078599989414215,
3048
+ "eval_runtime": 46.065,
3049
+ "eval_samples_per_second": 9.791,
3050
+ "eval_steps_per_second": 0.174,
3051
+ "step": 390
3052
+ },
3053
+ {
3054
+ "epoch": 9.768,
3055
+ "step": 390,
3056
+ "total_flos": 7.851213014748365e+18,
3057
+ "train_loss": 0.11308465492553436,
3058
+ "train_runtime": 13137.6351,
3059
+ "train_samples_per_second": 3.806,
3060
+ "train_steps_per_second": 0.03
3061
+ }
3062
+ ],
3063
+ "logging_steps": 1,
3064
+ "max_steps": 390,
3065
+ "num_input_tokens_seen": 0,
3066
+ "num_train_epochs": 10,
3067
+ "save_steps": 500,
3068
+ "stateful_callbacks": {
3069
+ "TrainerControl": {
3070
+ "args": {
3071
+ "should_epoch_stop": false,
3072
+ "should_evaluate": false,
3073
+ "should_log": false,
3074
+ "should_save": true,
3075
+ "should_training_stop": true
3076
+ },
3077
+ "attributes": {}
3078
+ }
3079
+ },
3080
+ "total_flos": 7.851213014748365e+18,
3081
+ "train_batch_size": 2,
3082
+ "trial_name": null,
3083
+ "trial_params": null
3084
+ }
CRM/sft-qwen2.5-math-prm-7b-score-model-simple-bs128/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e58bf2d34fe45c8cf3d1e4929a5e0a46db02f846085781f1d5c6c3d29d72f02a
3
+ size 5496