kevin009 commited on
Commit
b584b44
·
verified ·
1 Parent(s): 9abe956

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +23 -14
README.md CHANGED
@@ -1,22 +1,31 @@
1
  ---
2
- base_model: llama3-merged-adapters
3
- tags:
4
- - text-generation-inference
5
- - transformers
6
- - unsloth
7
- - llama
8
- - trl
9
  license: apache-2.0
10
  language:
11
  - en
 
 
 
 
 
 
12
  ---
13
 
14
- # Uploaded model
15
-
16
- - **Developed by:** kevin009
17
- - **License:** apache-2.0
18
- - **Finetuned from model :** llama3-merged-adapters
19
 
20
- This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
 
21
 
22
- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
 
 
 
 
 
 
 
2
  license: apache-2.0
3
  language:
4
  - en
5
+ base_model:
6
+ - meta-llama/Llama-3.1-8B-instruct
7
+ pipeline_tag: text-generation
8
+ tags:
9
+ - lora
10
+ - adapter
11
  ---
12
 
13
+ ## Model Details
14
+ - Base Model: meta-llama/Llama-3.1-8B-instruct
15
+ - KTO
 
 
16
 
17
+ ## Datasets:
18
+ - Revision CoT
19
 
20
+ ### Source Adapters
21
+ All source adapters share the following configuration:
22
+ - Rank (r): 16
23
+ - Alpha: 16
24
+ - Target Modules:
25
+ - q_proj (Query projection)
26
+ - k_proj (Key projection)
27
+ - v_proj (Value projection)
28
+ - o_proj (Output projection)
29
+ - up_proj (Upsampling projection)
30
+ - down_proj (Downsampling projection)
31
+ - gate_proj (Gate projection)