Hugging Face's logo

SicariusSicariiStuff
/
Negative_LLAMA_70B_EXL3_d_2.5bpw

Text Generation
Transformers
Safetensors
English
llama
conversational
text-generation-inference
exl3
Model card Files Community
Configuration Parsing Warning: In config.json: "quantization_config.bits" must be an integer

Note

This was made to test extremely low quants. 70B is pretty usable at 2.50 bpw. A custom strategy for quanting was used, for details check EXL3 repo. This was made to fit a single 32GB card like the 5090.

Downloads last month
10
Safetensors
Model size
12.3B params
Tensor type
F16
·
I16
·
Inference Providers NEW
Text Generation
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for SicariusSicariiStuff/Negative_LLAMA_70B_EXL3_d_2.5bpw

Base model

meta-llama/Llama-3.1-70B
Finetuned
meta-llama/Llama-3.3-70B-Instruct
Finetuned
SicariusSicariiStuff/Negative_LLAMA_70B
Quantized
(22)
this model

Dataset used to train SicariusSicariiStuff/Negative_LLAMA_70B_EXL3_d_2.5bpw