OddTheGreat commited on
Commit
f25780f
·
verified ·
1 Parent(s): 26fee01

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -39
README.md CHANGED
@@ -7,45 +7,33 @@ library_name: transformers
7
  tags:
8
  - mergekit
9
  - merge
10
-
 
 
 
 
 
11
  ---
12
  # merge
13
 
14
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
15
-
16
- ## Merge Details
17
- ### Merge Method
18
-
19
- This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [huihui-ai/Mistral-Small-24B-Instruct-2501-abliterated](https://huggingface.co/huihui-ai/Mistral-Small-24B-Instruct-2501-abliterated) as a base.
20
-
21
- ### Models Merged
22
-
23
- The following models were included in the merge:
24
- * [PocketDoc/Dans-DangerousWinds-V1.1.1-24b](https://huggingface.co/PocketDoc/Dans-DangerousWinds-V1.1.1-24b)
25
- * [trashpanda-org/MS-24B-Instruct-Mullein-v0](https://huggingface.co/trashpanda-org/MS-24B-Instruct-Mullein-v0)
26
-
27
- ### Configuration
28
-
29
- The following YAML configuration was used to produce this model:
30
-
31
- ```yaml
32
- models:
33
- - model: huihui-ai/Mistral-Small-24B-Instruct-2501-abliterated
34
- parameters:
35
- density: 0.7
36
- weight: 0.7
37
- - model: PocketDoc/Dans-DangerousWinds-V1.1.1-24b
38
- parameters:
39
- density: 0.5
40
- weight: 0.5
41
- - model: trashpanda-org/MS-24B-Instruct-Mullein-v0
42
- parameters:
43
- density: 0.3
44
- weight: 0.3
45
- merge_method: ties
46
- base_model: huihui-ai/Mistral-Small-24B-Instruct-2501-abliterated
47
- parameters:
48
- normalize: false
49
- int8_mask: true
50
- dtype: float16
51
- ```
 
7
  tags:
8
  - mergekit
9
  - merge
10
+ - dark
11
+ - roleplay
12
+ - negative
13
+ language:
14
+ - en
15
+ - ru
16
  ---
17
  # merge
18
 
19
+ This is a merge of pre-trained language models.
20
+
21
+ This merge is better than v1. v1 machina will be deleted tomorrow.
22
+
23
+ Has neutral or negative bias, if prompt is good enough model can be evil and cruel. maybe even too cruel.
24
+
25
+ Main idea is to merge base mistral with something, and preserve base's ability to write on russian.
26
+
27
+ Secondary idea was to create model that will be not "friendship is magic" thing, to use in more "dark" scenarios.
28
+
29
+ Judging by the tests, i've succeeded with both ideas.
30
+
31
+ Model was tested on russian and on english with ~1000 responses, was stable and seems that it not lose original's "intellectual abilities", but maybe i'm just very lucky.
32
+
33
+ With char cards that have first message on russian, but whole description on english, model have some difficulties, but still able to answer on desired language.
34
+
35
+ With full ru cards model performs without issues.
36
+
37
+ Tested on T1.01 ChatML
38
+
39
+ I reccomend to use sphiratrioth666/SillyTavern-Presets-Sphiratrioth, for me it works very good with minor adjustments.