Multimodal GGUFs Vision and audio models compatible with llama-server and llama-mtmd-cli Gemma 3 Collection 4 items • Updated about 7 hours ago • 17 Kimi-VL Collection 2 items • Updated about 7 hours ago ggml-org/Mistral-Small-3.1-24B-Instruct-2503-GGUF Image-Text-to-Text • 24B • Updated May 1 • 395 • 4 InternVL 3 and InternVL 2.5 Collection 10 items • Updated about 7 hours ago
GPT OSS ggml-org/gpt-oss-120b-GGUF 117B • Updated 11 days ago • 31.2k • 20 ggml-org/gpt-oss-20b-GGUF 21B • Updated 11 days ago • 77k • 63
Gemma 3n ggml-org/gemma-3n-E2B-it-GGUF 4B • Updated Jun 26 • 2.14k • 13 ggml-org/gemma-3n-E4B-it-GGUF 7B • Updated Jun 26 • 1.84k • 16
InternVL 3 and InternVL 2.5 ggml-org/InternVL3-1B-Instruct-GGUF 0.6B • Updated May 10 • 707 • 3 ggml-org/InternVL3-2B-Instruct-GGUF 2B • Updated May 10 • 685 • 5 ggml-org/InternVL3-8B-Instruct-GGUF 8B • Updated May 10 • 873 • 5 ggml-org/InternVL3-14B-Instruct-GGUF 15B • Updated May 10 • 674 • 3
Qwen 3 ggml-org/Qwen3-0.6B-GGUF 0.8B • Updated Apr 28 • 951 • 5 ggml-org/Qwen3-1.7B-GGUF 2B • Updated Apr 28 • 1.45k • 2 ggml-org/Qwen3-4B-GGUF 4B • Updated Apr 28 • 523 • 1 ggml-org/Qwen3-8B-GGUF 8B • Updated Apr 28 • 523 • 3
Gemma 3 ggml-org/gemma-3-1b-it-GGUF 1.0B • Updated Mar 12 • 12.2k • 19 ggml-org/gemma-3-4b-it-GGUF Image-Text-to-Text • 4B • Updated May 21 • 15.4k • 37 ggml-org/gemma-3-12b-it-GGUF Image-Text-to-Text • 12B • Updated May 21 • 2.89k • 27 ggml-org/gemma-3-27b-it-GGUF Image-Text-to-Text • 27B • Updated May 21 • 1.92k • 21
GGUF LoRA adapters Adapters extracted from fine tuned models, using mergekit-extract-lora ggml-org/LoRA-Llama-3-Instruct-abliteration-8B-F16-GGUF 0.1B • Updated Nov 1, 2024 • 16 ggml-org/LoRA-Qwen2.5-1.5B-Instruct-abliterated-F16-GGUF 0.1B • Updated Jan 23 • 28 • 2 ggml-org/LoRA-Qwen2.5-3B-Instruct-abliterated-F16-GGUF 0.1B • Updated Jan 9 • 12 • 1 ggml-org/LoRA-Qwen2.5-7B-Instruct-abliterated-v3-F16-GGUF 0.1B • Updated Jan 8 • 30 • 3
Gemma 1.1 GGUFs ggml-org/gemma-1.1-2b-it-Q8_0-GGUF 3B • Updated Apr 5, 2024 • 1.23k • 1 ggml-org/gemma-1.1-7b-it-Q8_0-GGUF 9B • Updated Apr 5, 2024 • 24 ggml-org/gemma-1.1-7b-it-Q4_K_M-GGUF 9B • Updated Apr 5, 2024 • 1.33k • 4
Gemma 3-270m Collection of models for Gemma 3-270m ggml-org/gemma-3-270m-GGUF 0.3B • Updated 6 days ago • 5.36k • 11 ggml-org/gemma-3-270m-it-GGUF 0.3B • Updated 5 days ago • 2.54k • 13 ggml-org/gemma-3-270m-qat-GGUF 0.3B • Updated 6 days ago • 602 • 6 ggml-org/gemma-3-270m-it-qat-GGUF 0.3B • Updated 5 days ago • 1.4k • 10
Kimi-VL ggml-org/Kimi-VL-A3B-Instruct-GGUF 16B • Updated about 8 hours ago • 51 • 2 ggml-org/Kimi-VL-A3B-Thinking-2506-GGUF 16B • Updated about 7 hours ago • 32 • 5
Qwen 2 VL and Qwen 2.5 VL ggml-org/Qwen2.5-VL-3B-Instruct-GGUF 3B • Updated Apr 30 • 4.73k • 5 ggml-org/Qwen2.5-VL-7B-Instruct-GGUF 8B • Updated Apr 30 • 2.88k • 7 ggml-org/Qwen2.5-VL-32B-Instruct-GGUF 33B • Updated May 15 • 746 • 3 ggml-org/Qwen2-VL-2B-Instruct-GGUF 2B • Updated Apr 30 • 750 • 2
SmolVLM GGUF ggml-org/SmolVLM2-2.2B-Instruct-GGUF 2B • Updated Apr 30 • 5.88k • 18 ggml-org/SmolVLM2-500M-Video-Instruct-GGUF 0.4B • Updated Apr 30 • 2.18k • 11 ggml-org/SmolVLM2-256M-Video-Instruct-GGUF 0.2B • Updated Apr 30 • 626 • 6 ggml-org/SmolVLM-Instruct-GGUF 2B • Updated Apr 30 • 614 • 7
llama.cpp presets Models that are used for presets in llama.cpp. ggml-org/gte-small-Q8_0-GGUF Sentence Similarity • 0.0B • Updated Feb 6 • 72 • 1 ggml-org/bge-small-en-v1.5-Q8_0-GGUF Feature Extraction • 0.0B • Updated Feb 6 • 170 • 2 ggml-org/e5-small-v2-Q8_0-GGUF Sentence Similarity • 0.0B • Updated Feb 6 • 25
llama.vim Recommended models for the llama.vim and llama.vscode plugins ggml-org/Qwen2.5-Coder-0.5B-Q8_0-GGUF Text Generation • 0.5B • Updated Jan 31 • 11.5k • 6 ggml-org/Qwen2.5-Coder-1.5B-Q8_0-GGUF Text Generation • 2B • Updated Oct 28, 2024 • 6.24k • 9 ggml-org/Qwen2.5-Coder-3B-Q8_0-GGUF Text Generation • 3B • Updated Nov 26, 2024 • 1.84k • 5 ggml-org/Qwen2.5-Coder-7B-Q8_0-GGUF Text Generation • 8B • Updated Oct 28, 2024 • 2.38k • 5
Multimodal GGUFs Vision and audio models compatible with llama-server and llama-mtmd-cli Gemma 3 Collection 4 items • Updated about 7 hours ago • 17 Kimi-VL Collection 2 items • Updated about 7 hours ago ggml-org/Mistral-Small-3.1-24B-Instruct-2503-GGUF Image-Text-to-Text • 24B • Updated May 1 • 395 • 4 InternVL 3 and InternVL 2.5 Collection 10 items • Updated about 7 hours ago
Gemma 3-270m Collection of models for Gemma 3-270m ggml-org/gemma-3-270m-GGUF 0.3B • Updated 6 days ago • 5.36k • 11 ggml-org/gemma-3-270m-it-GGUF 0.3B • Updated 5 days ago • 2.54k • 13 ggml-org/gemma-3-270m-qat-GGUF 0.3B • Updated 6 days ago • 602 • 6 ggml-org/gemma-3-270m-it-qat-GGUF 0.3B • Updated 5 days ago • 1.4k • 10
GPT OSS ggml-org/gpt-oss-120b-GGUF 117B • Updated 11 days ago • 31.2k • 20 ggml-org/gpt-oss-20b-GGUF 21B • Updated 11 days ago • 77k • 63
Kimi-VL ggml-org/Kimi-VL-A3B-Instruct-GGUF 16B • Updated about 8 hours ago • 51 • 2 ggml-org/Kimi-VL-A3B-Thinking-2506-GGUF 16B • Updated about 7 hours ago • 32 • 5
Gemma 3n ggml-org/gemma-3n-E2B-it-GGUF 4B • Updated Jun 26 • 2.14k • 13 ggml-org/gemma-3n-E4B-it-GGUF 7B • Updated Jun 26 • 1.84k • 16
InternVL 3 and InternVL 2.5 ggml-org/InternVL3-1B-Instruct-GGUF 0.6B • Updated May 10 • 707 • 3 ggml-org/InternVL3-2B-Instruct-GGUF 2B • Updated May 10 • 685 • 5 ggml-org/InternVL3-8B-Instruct-GGUF 8B • Updated May 10 • 873 • 5 ggml-org/InternVL3-14B-Instruct-GGUF 15B • Updated May 10 • 674 • 3
Qwen 2 VL and Qwen 2.5 VL ggml-org/Qwen2.5-VL-3B-Instruct-GGUF 3B • Updated Apr 30 • 4.73k • 5 ggml-org/Qwen2.5-VL-7B-Instruct-GGUF 8B • Updated Apr 30 • 2.88k • 7 ggml-org/Qwen2.5-VL-32B-Instruct-GGUF 33B • Updated May 15 • 746 • 3 ggml-org/Qwen2-VL-2B-Instruct-GGUF 2B • Updated Apr 30 • 750 • 2
Qwen 3 ggml-org/Qwen3-0.6B-GGUF 0.8B • Updated Apr 28 • 951 • 5 ggml-org/Qwen3-1.7B-GGUF 2B • Updated Apr 28 • 1.45k • 2 ggml-org/Qwen3-4B-GGUF 4B • Updated Apr 28 • 523 • 1 ggml-org/Qwen3-8B-GGUF 8B • Updated Apr 28 • 523 • 3
SmolVLM GGUF ggml-org/SmolVLM2-2.2B-Instruct-GGUF 2B • Updated Apr 30 • 5.88k • 18 ggml-org/SmolVLM2-500M-Video-Instruct-GGUF 0.4B • Updated Apr 30 • 2.18k • 11 ggml-org/SmolVLM2-256M-Video-Instruct-GGUF 0.2B • Updated Apr 30 • 626 • 6 ggml-org/SmolVLM-Instruct-GGUF 2B • Updated Apr 30 • 614 • 7
Gemma 3 ggml-org/gemma-3-1b-it-GGUF 1.0B • Updated Mar 12 • 12.2k • 19 ggml-org/gemma-3-4b-it-GGUF Image-Text-to-Text • 4B • Updated May 21 • 15.4k • 37 ggml-org/gemma-3-12b-it-GGUF Image-Text-to-Text • 12B • Updated May 21 • 2.89k • 27 ggml-org/gemma-3-27b-it-GGUF Image-Text-to-Text • 27B • Updated May 21 • 1.92k • 21
llama.cpp presets Models that are used for presets in llama.cpp. ggml-org/gte-small-Q8_0-GGUF Sentence Similarity • 0.0B • Updated Feb 6 • 72 • 1 ggml-org/bge-small-en-v1.5-Q8_0-GGUF Feature Extraction • 0.0B • Updated Feb 6 • 170 • 2 ggml-org/e5-small-v2-Q8_0-GGUF Sentence Similarity • 0.0B • Updated Feb 6 • 25
GGUF LoRA adapters Adapters extracted from fine tuned models, using mergekit-extract-lora ggml-org/LoRA-Llama-3-Instruct-abliteration-8B-F16-GGUF 0.1B • Updated Nov 1, 2024 • 16 ggml-org/LoRA-Qwen2.5-1.5B-Instruct-abliterated-F16-GGUF 0.1B • Updated Jan 23 • 28 • 2 ggml-org/LoRA-Qwen2.5-3B-Instruct-abliterated-F16-GGUF 0.1B • Updated Jan 9 • 12 • 1 ggml-org/LoRA-Qwen2.5-7B-Instruct-abliterated-v3-F16-GGUF 0.1B • Updated Jan 8 • 30 • 3
llama.vim Recommended models for the llama.vim and llama.vscode plugins ggml-org/Qwen2.5-Coder-0.5B-Q8_0-GGUF Text Generation • 0.5B • Updated Jan 31 • 11.5k • 6 ggml-org/Qwen2.5-Coder-1.5B-Q8_0-GGUF Text Generation • 2B • Updated Oct 28, 2024 • 6.24k • 9 ggml-org/Qwen2.5-Coder-3B-Q8_0-GGUF Text Generation • 3B • Updated Nov 26, 2024 • 1.84k • 5 ggml-org/Qwen2.5-Coder-7B-Q8_0-GGUF Text Generation • 8B • Updated Oct 28, 2024 • 2.38k • 5
Gemma 1.1 GGUFs ggml-org/gemma-1.1-2b-it-Q8_0-GGUF 3B • Updated Apr 5, 2024 • 1.23k • 1 ggml-org/gemma-1.1-7b-it-Q8_0-GGUF 9B • Updated Apr 5, 2024 • 24 ggml-org/gemma-1.1-7b-it-Q4_K_M-GGUF 9B • Updated Apr 5, 2024 • 1.33k • 4