---
base_model: mistralai/Mistral-Small-Instruct-2409
---

[EXL3](https://github.com/turboderp-org/exllamav3) quantization of [Mistral-Small-Instruct-2409](https://huggingface.co/mistralai/Mistral-Small-Instruct-2409), 4 bits per weight.

### HumanEval (argmax)

| Model                                                                                                            | Q4   | Q6   | Q8   | FP16 |
| ---------------------------------------------------------------------------------------------------------------- | ---- | ---- | ---- | ---- |
| [Mistral-Small-Instruct-2409-exl3-3bpw](https://huggingface.co/isogen/Mistral-Small-Instruct-2409-exl3-3bpw)     | 76.8 | 74.4 | 76.2 | 75.6 |
| [Mistral-Small-Instruct-2409-exl3-3.5bpw](https://huggingface.co/isogen/Mistral-Small-Instruct-2409-exl3-3.5bpw) | 73.8 | 75.6 | 75.0 | 75.6 |
| [Mistral-Small-Instruct-2409-exl3-4bpw](https://huggingface.co/isogen/Mistral-Small-Instruct-2409-exl3-4bpw)     | 78.7 | 78.7 | 79.3 | 79.3 |
| [Mistral-Nemo-Instruct-2407-exl3-4bpw](https://huggingface.co/isogen/Mistral-Nemo-Instruct-2407-exl3-4bpw)       | 74.4 | 72.6 | 73.2 | 72.0 |
| [Mistral-Nemo-Instruct-2407-exl3-6bpw](https://huggingface.co/isogen/Mistral-Nemo-Instruct-2407-exl3-6bpw)       | 70.7 | 69.5 | 69.5 | 68.9 |