Update README.md
Browse files
README.md
CHANGED
@@ -70,7 +70,15 @@ While the macro F1 scores across the 1-5 rating scale are relatively low due to
|
|
70 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/61c141342aac764ce1654e43/06siAXeIRjJGnlPehW562.png" width="600">
|
71 |
</div>
|
72 |
|
73 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
74 |
|
75 |
### Training hyperparameters
|
76 |
|
|
|
70 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/61c141342aac764ce1654e43/06siAXeIRjJGnlPehW562.png" width="600">
|
71 |
</div>
|
72 |
|
73 |
+
The table below shows Stack-Edu dataset statistics and MultiPL-E scores for the top 4 (in terms of size) programming languages. We use HumanEval for Python evaluation. For the ablation, we started from a mid-training checkpoint of SmolLM2 at 3T tokens which was trained primarily on web data, and perform linear annealing on 200B tokens, uniformly distributed across 15 of the most commonly used programming languages (~14B tokens each).
|
74 |
+
|
75 |
+
| Language | Before filtering (B tokens) | After filtering (B tokens) | MultiPL-E (Original → Filtered) |
|
76 |
+
|------------|-------------------------|---------------------|-------------------------------|
|
77 |
+
| Python | 50.6 | 21.8 | 20.7 → 25.6 |
|
78 |
+
| C++ | 69.7 | 16.0 | 16.7 → 24.8 |
|
79 |
+
| JavaScript | 45.3 | 11.1 | 18.2 → 22.4 |
|
80 |
+
| Java | 45.6 | 42.1 | 17.6 → 22.7 |
|
81 |
+
|
82 |
|
83 |
### Training hyperparameters
|
84 |
|