metadata
datasets:
- Marcus2112/minipile_density-proportioned
language:
- en
base_model:
- EleutherAI/pythia-160m-deduped
| Benchmark | Measure | 160M Density | 160M Density 2 Epochs | Percentage Difference in Means | |
|---|---|---|---|---|---|
| ARC-Challenge | acc | ↑ | 0.1920 ± 0.0115 | 0.1894 ± 0.0115 | -1.3542 |
| MMLU | acc | ↑ | 0.2295 ± 0.0035 | 0.2295 ± 0.0035 | 0.0000 |
| HellaSwag | acc | ↑ | 0.2604 ± 0.0044 | 0.2568 ± 0.0044 | -1.3825 |
| WinoGrande | acc | ↑ | 0.5201 ± 0.0140 | 0.5012 ± 0.0141 | -3.6339 |
| Lambada (OpenAI) | acc | ↑ | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 | - |
| Lambada (OpenAI) | perplexity | ↓ | 2099002.0912 ± 170652.6222 | 1587737.3755 ± 121555.3148 | -24.3575 |
| Lambada (Std) | acc | ↑ | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 | - |
| Lambada (Std) | perplexity | ↓ | 13347273.6076 ± 1997894.6360 | 8366924.7603 ± 713077.3579 | -37.3136 |
| BLiMP | acc | ↑ | 0.5501 ± 0.0017 | 0.5378 ± 0.0017 | -2.2360 |