Update README.md
Browse files
README.md
CHANGED
@@ -18,10 +18,17 @@ datasets:
|
|
18 |
|
19 |
## Description
|
20 |
|
21 |
-
This model was developed for use in a competition.
|
22 |
|
23 |
## Uses
|
24 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
25 |
```python
|
26 |
from unsloth import FastLanguageModel
|
27 |
from peft import PeftModel
|
@@ -86,3 +93,8 @@ with open(f'./llm-jp-3-13b-it_lora_20241216_output.jsonl', 'w', encoding='utf-8'
|
|
86 |
### Training Procedure
|
87 |
|
88 |
This model was fine-tuned using LoRA (Low-Rank Adaptation) to optimize training efficiency and minimize computational overhead while maintaining performance. The fine-tuning process utilized Japanese instruction data specifically designed for LLMs to enhance its capabilities in understanding and generating Japanese-language instructions.
|
|
|
|
|
|
|
|
|
|
|
|
18 |
|
19 |
## Description
|
20 |
|
21 |
+
This model was developed for use in a competition, specifically for [松尾研大規模言語モデル講座2024](https://weblab.t.u-tokyo.ac.jp/lecture/course-list/large-language-model/).
|
22 |
|
23 |
## Uses
|
24 |
|
25 |
+
```python
|
26 |
+
!pip install unsloth
|
27 |
+
!pip install --upgrade --no-cache-dir "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
|
28 |
+
!pip install -U torch
|
29 |
+
!pip install -U peft
|
30 |
+
```
|
31 |
+
|
32 |
```python
|
33 |
from unsloth import FastLanguageModel
|
34 |
from peft import PeftModel
|
|
|
93 |
### Training Procedure
|
94 |
|
95 |
This model was fine-tuned using LoRA (Low-Rank Adaptation) to optimize training efficiency and minimize computational overhead while maintaining performance. The fine-tuning process utilized Japanese instruction data specifically designed for LLMs to enhance its capabilities in understanding and generating Japanese-language instructions.
|
96 |
+
|
97 |
+
This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
|
98 |
+
|
99 |
+
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|
100 |
+
|