josephmayo/von-vibethinker-3b-trackio-bucket
49.9 MB
josephmayo PRO
josephmayo
AI & ML interests
why dont we make smart models that can fit on consumer hardware
Recent Activity
updated a bucket 1 day ago
josephmayo/von-vibethinker-3b-trackio-bucket updated a Space 6 days ago
josephmayo/von-vibethinker-3b-trackio published a bucket 6 days ago
josephmayo/von-vibethinker-3b-trackio-bucketOrganizations
None yet
updated a
bucket 1 day ago
updated a
Space 6 days ago
published a
bucket 6 days ago
published a
Space 6 days ago
liked a
dataset 7 days ago
liked a
model 8 days ago
liked a
model 26 days ago
liked a
dataset 28 days ago
liked a
model about 1 month ago
liked a
dataset about 1 month ago
reacted to sergiopaniego's post with 👍 about 1 month ago
Post
7770
Frontier models use distillation as a step of their post-training pipelines.
In 2026 it has three jobs: compress a big model into a small one, merge RL experts into a single model, and let a model teach itself.
I wrote up which frontier models use each one and how: https://huggingface.co/blog/sergiopaniego/distillation-2026
It pairs with Class 2 of the Training an Agent series Ben and I are doing, where we teach these techniques hands-on with TRL!
In 2026 it has three jobs: compress a big model into a small one, merge RL experts into a single model, and let a model teach itself.
I wrote up which frontier models use each one and how: https://huggingface.co/blog/sergiopaniego/distillation-2026
It pairs with Class 2 of the Training an Agent series Ben and I are doing, where we teach these techniques hands-on with TRL!
- 3 replies
Donation cost to run Heretic or otherwise Abliterate this?
3
#3 opened about 2 months ago
by 
dissociativity
updated a
model about 2 months ago