Pruned Qwen (Epoch 1)
This is ToastyPigeon/qwen2.5-32b-unnamed-test-model pruned down from 32b -> 27b.
Using PruneMe to find layers to remove resulted in the removal of layers [25, 29) and [36, 43) for a reduction from 64 -> 52 layers.
Trained on 1 epoch of mixed data from the datasets that went into the pre-pruned model (I'll document that later), totaling about ~10M tokens so far of retraining.
Coherent but a little dumb. Will likely run a second epoch on this.
- Downloads last month
- 6
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model has no library tag.