---
license: other
license_name: modified-mit
library_name: mlx
base_model: moonshotai/Kimi-K2-Instruct
pipeline_tag: text-generation
tags:
- mlx
---
**See Kimi-K2 Dynamic MLX in action - [https://youtu.be/-zfUvA2CDqE](https://youtu.be/-zfUvA2CDqE)**

*q3.95bit dynamic quant achieves 1.243 perplexity in our testing, slotting closer to q4 perplexity (1.168) than q3 perplexity (1.900).*
| Quantization | Perplexity |
|:------------:|:----------:|
| **q2**       | 41.293     |
| **q3**       | 1.900      |
| **q3.95**    | 1.243      |
| **q4**       | 1.168      |
| **q6**       | 1.128      |
| **q8**       | 1.128      |

## Usage Notes

* Built with a modified version of [MLX](https://github.com/ml-explore/mlx) 0.26
* Runs on a single M3 Ultra 512GB RAM
* Requires expanding VRAM limit to at least ~500000 MB
  * For a larger context window, 507000 is used in VRAM limit command below.
  * `sudo sysctl iogpu.wired_limit_mb=507000`
* Expect ~20 tokens/s
* For more details see [demonstration video](https://youtu.be/-zfUvA2CDqE) or visit [Kimi K2](https://moonshotai.github.io/Kimi-K2/).