Update README.md
Browse files
README.md
CHANGED
|
@@ -1,5 +1,10 @@
|
|
| 1 |
---
|
| 2 |
license: cc-by-nc-4.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
| 4 |
|
| 5 |
```
|
|
@@ -22,4 +27,35 @@ PROUDLY PRESENTS
|
|
| 22 |
|
| 23 |
# Phainesthesia-8x7B-iMat-GGUF
|
| 24 |
|
| 25 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: cc-by-nc-4.0
|
| 3 |
+
tags:
|
| 4 |
+
- not-for-all-audiences
|
| 5 |
+
- GGUF
|
| 6 |
+
- merge
|
| 7 |
+
- iMat
|
| 8 |
---
|
| 9 |
|
| 10 |
```
|
|
|
|
| 27 |
|
| 28 |
# Phainesthesia-8x7B-iMat-GGUF
|
| 29 |
|
| 30 |
+
Quantized from fp16 with love.
|
| 31 |
+
* Quantizations made possible using .imatrix file from [this](https://huggingface.co/datasets/ikawrakow/imatrix-from-wiki-train) repo (special thanks to [ikawrakow](https://huggingface.co/ikawrakow) again)
|
| 32 |
+
|
| 33 |
+
For a brief rundown of iMatrix quant performance please see this [PR](https://github.com/ggerganov/llama.cpp/pull/5747)
|
| 34 |
+
|
| 35 |
+
<i>All quants are verified working prior to uploading to repo for your safety and convenience. </i>
|
| 36 |
+
|
| 37 |
+
Please note importance matrix quantizations are a work in progress, IQ3 and above is recommended for best results.
|
| 38 |
+
|
| 39 |
+
<b>Tip:</b> Pick a size that can fit in your GPU while still allowing some room for context for best speed. You may need to pad this further depending on if you are running image gen or TTS as well.
|
| 40 |
+
|
| 41 |
+
Original model card can be found [here](https://huggingface.co/Envoid/Phainesthesia-8x7B) and below.
|
| 42 |
+
|
| 43 |
+
|
| 44 |
+
---
|
| 45 |
+
# Warning this model is highly experimental and unpredictable.
|
| 46 |
+
|
| 47 |
+
|
| 48 |
+
|
| 49 |
+
Phainesthesia-8x7B began its life as a copy of [Verdict-DADA-8x7B](https://huggingface.co/Envoid/Verdict-DADA-8x7B)
|
| 50 |
+
|
| 51 |
+
In a training configuration accident it gained some rather uncanny abilities but was extremely disorderly. Further experimentation was done to decide upon a course of action going forward.
|
| 52 |
+
|
| 53 |
+
It then received a corrective qlora session to attempt to correct the disorder.
|
| 54 |
+
|
| 55 |
+
It hasn't been tested on long roleplay scenarios yet. Although testing with simple temperatures with t=1 while requiring frequent rerolls the messages that are good are good, although it is constantly walking the razor's edge with looping on one side and a complete loss of coherence on the other (with the edge of the razor being pure gold)
|
| 56 |
+
|
| 57 |
+
Its assistant personality on determinstic sampling is rather run of the mill but with simple sampling at t=1 It gives much more interesting replies and is prone to hallucination when asking leading questions. Others might refer to this as an imagination.
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
|
| 61 |
+
### Only tested in Q8 (not included)
|