Predicting the Order of Upcoming Tokens Improves Language Modeling
Paper
•
2508.19228
•
Published
•
21
Token Order Prediction
Pretrained models from the paper "Predicting the Order of Upcoming Tokens Improves Language Modeling"
-
-
zaydzuhri/vanilla-340M-4096-model
0.4B • Updated • 121 -
zaydzuhri/mtp-340M-4096-model
0.4B • Updated • 109 -
zaydzuhri/top-340M-4096-model
0.4B • Updated • 40 • 1 -
zaydzuhri/vanilla-1.8B-4096-model
2B • Updated • 62 -
zaydzuhri/mtp-1.8B-4096-model
2B • Updated • 78 -
zaydzuhri/top-1.8B-4096-model
2B • Updated • 79 -
zaydzuhri/vanilla-7B-4096-model
7B • Updated • 76 -
zaydzuhri/mtp-7B-4096-model
7B • Updated • 82 -
zaydzuhri/top-7B-4096-model
7B • Updated • 67