Tom-Neverwinter/ew-lora
Updated
•
4
s
Tom-Neverwinter
AI & ML interests
Making improvements to help the world.
Recent Activity
new activity
4 days ago
perplexity-ai/r1-1776:Was this Model Needed?
reacted
to
csabakecskemeti's
post
with 🔥
about 2 months ago
I've built a small utility to split safetensors file by file.
The issue/need came up when I've tried to convert the new Deepseek V3 model from FP8 to BF16.
The only Ada architecture GPU I have is an RTX 4080 and the 16GB vram was just wasn't enough for the conversion.
BTW: I'll upload the bf16 version here:
https://huggingface.co/DevQuasar/deepseek-ai.DeepSeek-V3-Base-bf16
(it will take a while - days with my upload speed)
If anyone has access the resources to test it I'd appreciate a feedback if it's working or not.
The tool, is available from here:
https://github.com/csabakecskemeti/ai_utils/blob/main/safetensor_splitter.py
It's splitting every file to n pieces by the layers if possible, and create a new "model.safetensors.index.json" file.
I've tested it with Llama 3.1 8B and multiple split sizes, and validated by using inference pipeline.
use `--help` for usage
Please note current version expects the model is already multiple file and have a "model.safetensors.index.json" layer-safetensor mapping file.
Organizations
None yet
models
4
datasets
None public yet