README.md

metadata

base_model:
  - arcee-train/arcee-maz-mistral-24b-v3
library_name: transformers
tags:
  - mergekit
  - merge

Mistral-24B-MazMerge-V3

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the Arcee Fusion merge method using /workspace/models/Mistral-24B-MazMerge-V1 as a base.

Models Merged

The following models were included in the merge:

• arcee-train/arcee-maz-mistral-24b-v3

Configuration

The following YAML configuration was used to produce this model:

base_model: /workspace/models/Mistral-24B-MazMerge-V1
dtype: float32
merge_method: arcee_fusion
out_dtype: bfloat16
slices:
- sources:
  - layer_range: [0, 40]
    model: arcee-train/arcee-maz-mistral-24b-v3
  - layer_range: [0, 40]
    model: /workspace/models/Mistral-24B-MazMerge-V1
tokenizer:
  source: arcee-train/arcee-maz-mistral-24b-v3