Unsloth 4-bit Dynamic Quants - a unsloth Collection

unsloth 's Collections

Unsloth 4-bit Dynamic Quants

DeepSeek R1 (All Versions)

Deepseek V3 (All Versions)

Phi-4 (All Versions)

Mistral-Small-24B-2501 (All Versions)

Llama 3.3 (All Versions)

Qwen2.5-VL (All Versions)

Qwen QVQ + QwQ Collection

Vision/multimodal Models

Llama 3.2 Vision

Llama 3.1 Collection

Load 4bit models 4x faster

4bit Instruct Models

Unsloth 4-bit Dynamic Quants

updated 8 days ago

Unsloths Dynamic 4bit Quants selectively skips quantizing certain parameters; greatly improving accuracy while only using <10% more VRAM than BnB 4bit