metadata
license: other
license_name: modified-mit
library_name: mlx
base_model: moonshotai/Kimi-K2-Instruct
pipeline_tag: text-generation
tags:
- mlx
See Kimi-K2 Dynamic MLX in action - https://youtu.be/-zfUvA2CDqE
q3.95bit dynamic quant achieves 1.243 perplexity in our testing, slotting closer to q4 perplexity (1.168) than q3 perplexity (1.900).
| Quantization | Perplexity |
|---|---|
| q2 | 41.293 |
| q3 | 1.900 |
| q3.95 | 1.243 |
| q4 | 1.168 |
| q6 | 1.128 |
| q8 | 1.128 |
Usage Notes
- Built with a modified version of MLX 0.26
- Runs on a single M3 Ultra 512GB RAM
- Requires expanding VRAM limit to at least ~500000 MB
- For a larger context window, 507000 is used in VRAM limit command below.
sudo sysctl iogpu.wired_limit_mb=507000
- Expect ~20 tokens/s
- For more details see demonstration video or visit Kimi K2.