Update README.md
Browse files
README.md
CHANGED
@@ -99,10 +99,11 @@ snapshot_download(
|
|
99 |
merged_file.gguf
|
100 |
```
|
101 |
|
102 |
-
|
|
103 |
| -------- | -------- | ------------ | ------------ | ---------------------| ---------- |
|
104 |
-
|
|
105 |
-
|
|
|
|
106 |
|
107 |
# Finetune your own Reasoning model like R1 with Unsloth!
|
108 |
We have a free Google Colab notebook for turning Llama 3.1 (8B) into a reasoning model: https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.1_(8B)-GRPO.ipynb
|
|
|
99 |
merged_file.gguf
|
100 |
```
|
101 |
|
102 |
+
| Dynamic Bits | Type | Disk Size | Accuracy | Link | Details |
|
103 |
| -------- | -------- | ------------ | ------------ | ---------------------| ---------- |
|
104 |
+
| 2bit | UD-Q2_K_XL | **211GB** | Better | [Link](https://huggingface.co/unsloth/r1-1776-GGUF/tree/main/r1-1776-UD-Q2_K_XL) | MoE all 2.5bit. `down_proj` in MoE mixture of 3.5/2.5bit |
|
105 |
+
| 3bit | UD-Q3_K_XL | **298GB** | Best | [Link](https://huggingface.co/unsloth/r1-1776-GGUF/tree/main/r1-1776-UD-Q3_K_XL) | MoE Q3_K_M. Attention parts are upcasted |
|
106 |
+
| 4bit | UD-Q4_K_XL | **377GB** | Best | [Link](https://huggingface.co/unsloth/r1-1776-GGUF/tree/main/r1-1776-UD-Q4_K_XL) | MoE Q4_K_M. Attention parts are upcasted |
|
107 |
|
108 |
# Finetune your own Reasoning model like R1 with Unsloth!
|
109 |
We have a free Google Colab notebook for turning Llama 3.1 (8B) into a reasoning model: https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.1_(8B)-GRPO.ipynb
|