Teera commited on
Commit
35a8be3
·
verified ·
1 Parent(s): d5916be

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +36 -6
README.md CHANGED
@@ -9,14 +9,44 @@ tags:
9
  license: apache-2.0
10
  language:
11
  - en
 
12
  ---
13
 
14
- # Uploaded model
15
 
16
- - **Developed by:** Teera
17
- - **License:** apache-2.0
18
- - **Finetuned from model :** Xkev/Llama-3.2V-11B-cot
19
 
20
- This mllama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
21
 
22
- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
  license: apache-2.0
10
  language:
11
  - en
12
+ - th
13
  ---
14
 
15
+ # Model Card for Model ID
16
 
17
+ Teera/Llama-3.2v-COT-Thai is a fine-tuned model based on Llama-3.2V-11B-co, developed with inspiration from the LLaVA-CoT framework.
 
 
18
 
19
+ The concept was introduced in **LLaVA-CoT: Let Vision Language Models Reason Step-by-Step.**
20
 
21
+ ## Training Details
22
+
23
+ # Training Data
24
+
25
+ The model is trained on the LLaVA-CoT-100k dataset, preprocess and translat to thai language.
26
+
27
+ # Training Procedure
28
+ The model is finetuned on llama-recipes with the following settings. Using the same setting should accurately reproduce our results.
29
+
30
+ | Parameter | Value |
31
+ |-------------------------------|---------------------------------------------------|
32
+ | FSDP | enabled |
33
+ | lr | 1e-4 |
34
+ | num_epochs | 1 |
35
+ | batch_size_training | 2 |
36
+ | use_fast_kernels | True |
37
+ | run_validation | False |
38
+ | batching_strategy | padding |
39
+ | context_length | 4096 |
40
+ | gradient_accumulation_steps | 1 |
41
+ | gradient_clipping | False |
42
+ | gradient_clipping_threshold | 1.0 |
43
+ | weight_decay | 0.0 |
44
+ | gamma | 0.85 |
45
+ | seed | 42 |
46
+ | use_fp16 | False |
47
+ | mixed_precision | True |
48
+
49
+
50
+ ## Bias, Risks, and Limitations
51
+
52
+ The model may generate biased or offensive content, similar to other VLMs, due to limitations in the training data. Technically, the model's performance in aspects like instruction following still falls short of leading industry models.