File size: 1,278 Bytes
989852e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
---
base_model: mistralai/Mistral-Small-Instruct-2409
---
[EXL3](https://github.com/turboderp-org/exllamav3) quantization of [Mistral-Small-Instruct-2409](https://huggingface.co/mistralai/Mistral-Small-Instruct-2409), 4 bits per weight.
### HumanEval (argmax)
| Model | Q4 | Q6 | Q8 | FP16 |
| ---------------------------------------------------------------------------------------------------------------- | ---- | ---- | ---- | ---- |
| [Mistral-Small-Instruct-2409-exl3-3bpw](https://huggingface.co/isogen/Mistral-Small-Instruct-2409-exl3-3bpw) | 76.8 | 74.4 | 76.2 | 75.6 |
| [Mistral-Small-Instruct-2409-exl3-3.5bpw](https://huggingface.co/isogen/Mistral-Small-Instruct-2409-exl3-3.5bpw) | 73.8 | 75.6 | 75.0 | 75.6 |
| [Mistral-Small-Instruct-2409-exl3-4bpw](https://huggingface.co/isogen/Mistral-Small-Instruct-2409-exl3-4bpw) | 78.7 | 78.7 | 79.3 | 79.3 |
| [Mistral-Nemo-Instruct-2407-exl3-4bpw](https://huggingface.co/isogen/Mistral-Nemo-Instruct-2407-exl3-4bpw) | 74.4 | 72.6 | 73.2 | 72.0 |
| [Mistral-Nemo-Instruct-2407-exl3-6bpw](https://huggingface.co/isogen/Mistral-Nemo-Instruct-2407-exl3-6bpw) | 70.7 | 69.5 | 69.5 | 68.9 |
|