danielhanchen commited on
Commit
127dd4a
·
verified ·
1 Parent(s): 61e374f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -3
README.md CHANGED
@@ -99,10 +99,11 @@ snapshot_download(
99
  merged_file.gguf
100
  ```
101
 
102
- | MoE Bits | Type | Disk Size | Accuracy | Link | Details |
103
  | -------- | -------- | ------------ | ------------ | ---------------------| ---------- |
104
- | 2.22bit | UD-IQ2_XXS | **183GB** | Better | [Link](https://huggingface.co/unsloth/r1-1776-GGUF/tree/main/r1-1776-UD-IQ2_XXS) | MoE all 2.06bit. `down_proj` in MoE mixture of 2.5/2.06bit |
105
- | 2.51bit | UD-Q2_K_XL | **212GB** | Best | [Link](https://huggingface.co/unsloth/r1-1776-GGUF/tree/main/r1-1776-UD-Q2_K_XL) | MoE all 2.5bit. `down_proj` in MoE mixture of 3.5/2.5bit |
 
106
 
107
  # Finetune your own Reasoning model like R1 with Unsloth!
108
  We have a free Google Colab notebook for turning Llama 3.1 (8B) into a reasoning model: https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.1_(8B)-GRPO.ipynb
 
99
  merged_file.gguf
100
  ```
101
 
102
+ | Dynamic Bits | Type | Disk Size | Accuracy | Link | Details |
103
  | -------- | -------- | ------------ | ------------ | ---------------------| ---------- |
104
+ | 2bit | UD-Q2_K_XL | **211GB** | Better | [Link](https://huggingface.co/unsloth/r1-1776-GGUF/tree/main/r1-1776-UD-Q2_K_XL) | MoE all 2.5bit. `down_proj` in MoE mixture of 3.5/2.5bit |
105
+ | 3bit | UD-Q3_K_XL | **298GB** | Best | [Link](https://huggingface.co/unsloth/r1-1776-GGUF/tree/main/r1-1776-UD-Q3_K_XL) | MoE Q3_K_M. Attention parts are upcasted |
106
+ | 4bit | UD-Q4_K_XL | **377GB** | Best | [Link](https://huggingface.co/unsloth/r1-1776-GGUF/tree/main/r1-1776-UD-Q4_K_XL) | MoE Q4_K_M. Attention parts are upcasted |
107
 
108
  # Finetune your own Reasoning model like R1 with Unsloth!
109
  We have a free Google Colab notebook for turning Llama 3.1 (8B) into a reasoning model: https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.1_(8B)-GRPO.ipynb