Upload README.md
Browse files
README.md
CHANGED
@@ -21,7 +21,7 @@ This quant was made using exllamav2-0.0.21 with [pippa dataset](https://huggingf
|
|
21 |
|
22 |
This quant fits over 20k context on 24GB VRAM on Windows in my local testing (with exl2 Q4 cache), you might be able to get more depending on other things taking VRAM.
|
23 |
|
24 |
-
I tested this quant shortly in some random RPs (including
|
25 |
|
26 |
## Prompt Templates
|
27 |
|
@@ -35,7 +35,7 @@ Further details on prompting this model will also pop up under the [model discus
|
|
35 |
|
36 |
2.4bpw exl2 quant on default dataset: [Midnight-Miqu-70B-v1.5_exl2_2.4bpw](https://huggingface.co/DeusImperator/Midnight-Miqu-70B-v1.5_exl2_2.4bpw)
|
37 |
|
38 |
-
The above quant might be a little smarter based on limited testing, but this rpcal one might be a bit better for RP
|
39 |
|
40 |
### Original readme below
|
41 |
|
|
|
21 |
|
22 |
This quant fits over 20k context on 24GB VRAM on Windows in my local testing (with exl2 Q4 cache), you might be able to get more depending on other things taking VRAM.
|
23 |
|
24 |
+
I tested this quant shortly in some random RPs (including ones over 8k and 20k context) and it seems to work fine.
|
25 |
|
26 |
## Prompt Templates
|
27 |
|
|
|
35 |
|
36 |
2.4bpw exl2 quant on default dataset: [Midnight-Miqu-70B-v1.5_exl2_2.4bpw](https://huggingface.co/DeusImperator/Midnight-Miqu-70B-v1.5_exl2_2.4bpw)
|
37 |
|
38 |
+
The above quant might be a little smarter based on limited testing, but this rpcal one might be a bit better for RP.
|
39 |
|
40 |
### Original readme below
|
41 |
|