Configuration Parsing Warning: In config.json: "quantization_config.bits" must be an integer

Configuration Parsing Warning: In config.json: "quantization_config.bits" must be greater than or equal to 2

Note

This was made to test extremely low quants. 70B is NOT usable at 1.6 bpw. A custom strategy for quanting was used, for details check EXL3 repo.

Safetensors

Model size

8.3B params

Tensor type

F16

I16

Model tree for SicariusSicariiStuff/Negative_LLAMA_70B_EXL3_1.6bpw

Base model

Finetuned

Finetuned

Quantized

(22)

this model