README.md

---
library_name: transformers
tags:
- not-for-all-audiences
model-index:
- name: OpenCrystal-12B-L3
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: IFEval (0-Shot)
      type: HuggingFaceH4/ifeval
      args:
        num_few_shot: 0
    metrics:
    - type: inst_level_strict_acc and prompt_level_strict_acc
      value: 40.71
      name: strict accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Darkknight535/OpenCrystal-12B-L3
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: BBH (3-Shot)
      type: BBH
      args:
        num_few_shot: 3
    metrics:
    - type: acc_norm
      value: 31.84
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Darkknight535/OpenCrystal-12B-L3
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MATH Lvl 5 (4-Shot)
      type: hendrycks/competition_math
      args:
        num_few_shot: 4
    metrics:
    - type: exact_match
      value: 7.93
      name: exact match
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Darkknight535/OpenCrystal-12B-L3
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GPQA (0-shot)
      type: Idavidrein/gpqa
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 7.49
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Darkknight535/OpenCrystal-12B-L3
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MuSR (0-shot)
      type: TAUR-Lab/MuSR
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 5.74
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Darkknight535/OpenCrystal-12B-L3
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MMLU-PRO (5-shot)
      type: TIGER-Lab/MMLU-Pro
      config: main
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 29.34
      name: accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Darkknight535/OpenCrystal-12B-L3
      name: Open LLM Leaderboard
---
### OpenCrystal-12B-L3
This is a finetuned language model. (I recommend using this one v2 and v2.1 are not good enough)

![Rohma](https://huggingface.co/Darkknight535/OpenCrystal-12B-L3/resolve/main/1s.jpg)

### 128K??
[L3.1 Variant here](https://huggingface.co/Darkknight535/OpenCrystal-12B-L3.1-128K)

### Instruct Template
Default llama3 instruct and context preset, but here is the one i use.
[Instruct](https://huggingface.co/Darkknight535/OpenCrystal-12B-L3/blob/main/Llama%203%20%5BInstruct%5D.json)
[Context](https://huggingface.co/Darkknight535/OpenCrystal-12B-L3/blob/main/Llama%203%20%5BContext%5D.json)

### Samplers


## Creative
```
Temp : 1.23
Min P : 0.05
Repetition Penalty : 1.05

[And everything else neutral]
```

## Normal
```
Temp : 0.6 - 0.8
Min P : 0.1
Repetition Penalty : 1.1

[And everything else neutral]
```


### Pro Tip
- You can uncheck *Include Names* option in sillytavern, to force it to speak as others dynamically. **Not Recommended**
### Features

- Can speak as other npc automatically.
- Creative (Swipes are crazy.)
- Coherent (Sometime gets horny)
- Output feels like you're using Character.ai
- Follows prompt better
- Likes higher context length. (12K easily tested)
- can summarize and generate image prompts well [The Above image's prompt is generated in a roleplay by this model] (Possible : Due to llama-3-instruct as base)


### Instruct Prompt
```
You're {{char}}, follow {{char}} personality and plot of the story, Don't impersonate as {{user}}, Speak as others NPC except {{user}} when needed. Be Creative, Create various interesting events and situations during the story.
```

### FeedBack
[FeedBack here](https://huggingface.co/Darkknight535/OpenCrystal-12B-L3/discussions/1)
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_Darkknight535__OpenCrystal-12B-L3)

|      Metric       |Value|
|-------------------|----:|
|Avg.               |20.51|
|IFEval (0-Shot)    |40.71|
|BBH (3-Shot)       |31.84|
|MATH Lvl 5 (4-Shot)| 7.93|
|GPQA (0-shot)      | 7.49|
|MuSR (0-shot)      | 5.74|
|MMLU-PRO (5-shot)  |29.34|