|
|
--- |
|
|
base_model: mistralai/Mistral-Small-Instruct-2409 |
|
|
--- |
|
|
|
|
|
[EXL3](https://github.com/turboderp-org/exllamav3) quantization of [Mistral-Small-Instruct-2409](https://huggingface.co/mistralai/Mistral-Small-Instruct-2409), 4 bits per weight. |
|
|
|
|
|
### HumanEval (argmax) |
|
|
|
|
|
| Model | Q4 | Q6 | Q8 | FP16 | |
|
|
| ---------------------------------------------------------------------------------------------------------------- | ---- | ---- | ---- | ---- | |
|
|
| [Mistral-Small-Instruct-2409-exl3-3bpw](https://huggingface.co/isogen/Mistral-Small-Instruct-2409-exl3-3bpw) | 76.8 | 74.4 | 76.2 | 75.6 | |
|
|
| [Mistral-Small-Instruct-2409-exl3-3.5bpw](https://huggingface.co/isogen/Mistral-Small-Instruct-2409-exl3-3.5bpw) | 73.8 | 75.6 | 75.0 | 75.6 | |
|
|
| [Mistral-Small-Instruct-2409-exl3-4bpw](https://huggingface.co/isogen/Mistral-Small-Instruct-2409-exl3-4bpw) | 78.7 | 78.7 | 79.3 | 79.3 | |
|
|
| [Mistral-Nemo-Instruct-2407-exl3-4bpw](https://huggingface.co/isogen/Mistral-Nemo-Instruct-2407-exl3-4bpw) | 74.4 | 72.6 | 73.2 | 72.0 | |
|
|
| [Mistral-Nemo-Instruct-2407-exl3-6bpw](https://huggingface.co/isogen/Mistral-Nemo-Instruct-2407-exl3-6bpw) | 70.7 | 69.5 | 69.5 | 68.9 | |
|
|
|
