Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
Paper
•
2304.01373
•
Published
•
9
Pythia Scaling Suite
Pythia is the first LLM suite designed specifically to enable scientific research on LLMs. To learn more see https://github.com/EleutherAI/pythia
Note The paper introduced the Pythia model suite
EleutherAI/pythia-14m
Text Generation • 39.2M • Updated • 38.4k • 26Note This model was trained after the paper was released.
EleutherAI/pythia-70m
95.6M • Updated • 92k • 76Note This model was trained after the paper was released.
-
EleutherAI/pythia-160m
Text Generation • 0.2B • Updated • 92.6k • 36 -
EleutherAI/pythia-410m
Text Generation • 0.5B • Updated • 38.4k • 36 -
EleutherAI/pythia-1b
Text Generation • 1B • Updated • 27.9k • 42 -
EleutherAI/pythia-1.4b
Text Generation • 2B • Updated • 15.4k • 26 -
EleutherAI/pythia-2.8b
Text Generation • 3B • Updated • 23.5k • 32 -
EleutherAI/pythia-6.9b
Text Generation • 7B • Updated • 15.2k • 58 -
EleutherAI/pythia-12b
Text Generation • 12B • Updated • 10.3k • 141 -
EleutherAI/pythia-70m-deduped
Text Generation • 95.6M • Updated • 170k • 27 -
EleutherAI/pythia-160m-deduped
Text Generation • 0.2B • Updated • 53.2k • 3 -
EleutherAI/pythia-410m-deduped
Text Generation • 0.5B • Updated • 8.99k • 20 -
EleutherAI/pythia-1b-deduped
Text Generation • 1B • Updated • 7.26k • 19 -
EleutherAI/pythia-1.4b-deduped
Text Generation • Updated • 6.42k • 21 -
EleutherAI/pythia-2.8b-deduped
Text Generation • 3B • Updated • 5.63k • 14 -
EleutherAI/pythia-6.9b-deduped
Text Generation • Updated • 6.32k • 8 -
EleutherAI/pythia-12b-deduped
Text Generation • Updated • 3.19k • 52