Confused about all the files in a LLM Model
For example inmistralai/Mistral-7B-Instruct-v0.3, there is :model-00001-of-00003.safetensors

model-00002-of-00003.safetensors

model-00003-of-00003.safetensorsand there is also theconsolidated.safetensorswhich looks to be a “concatentation” of all the other safetensors. What are all these safetensors for? Which one is actually the model that I want to use if that makes any sense? Do I need all the safetensors for finetuning my own model?
2024-11-29T04:25:16Z
To put it simply, the file is too large to handle over a network, so it is just split up. If you are using it with a library provided by HF, etc., it will be automatically concatenated, so there is no need to worry too much. If you want to do things on your own, please refer to the page below.Do I need all the safetensors for finetuning my own model?Yes.huggingface.coSerializationWe’re on a journey to advance and democratize artificial intelligence through open source and open science.
2024-11-29T04:51:21Z
I see. If HF is going to concatenate the sharded model anyways, why should I downloadconsolidated.safetensors? Or am I misunderstanding something?
2024-11-29T05:40:56Z
The linking is done by the program in memory, and the files are usually left divided. As long as you remember that HF’s model is based on folders, you won’t make any mistakes.I don’t know the background of this, as I haven’t been on HF for that long, but I think it was easier to manage HF’s disks and git at the time if they were divided into 5GB or 10GB units. Also, if you make a mistake with a 5GB transfer, you can just download it again, but if you make a mistake with a 100GB transfer, you’ve lost 100GB. Now there are models with 50GB partitions, so I think it’s probably fine even if it’s bigger, but well, there’s not much point in changing something that’s working.