Performance problems with finetuned model (Llama 2 7B based)
Hello,I am trying to finetune Llama 2 7B to enhance its function calling abilities. I am not sure what I am doing wrong, but the model is not really converging while training and the performance, hence, isn’t good at all.I am usingthisJupyter Notebook and tried it with and without quantization.These are the training and eval/testdatasets. I made them, and they are designed to learn the ‘get_current_weather’ function.As my datasets are small, I picked higher values for r and lora_alphar=256, lora_alpha=512Does anyone have any ideas on how to improve the model? Am I doing something wrong?I am very new to the topic of finetuning LLMs. So, any feedback or help would be very much appreciated.
2024-06-10T09:24:20Z
Hi,i’d recommend taking a look at the following resources to fine-tune open-source LLMs:Unsloth:GitHub - unslothai/unsloth: Finetune Llama 3, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory. I see they recommend lower values for lora:Google Colab.the Alignment handbook (optimized for chat fine-tuning):alignment-handbook/scripts at main · huggingface/alignment-handbook · GitHub.I’d recommend starting with a super tiny dataset and see if the model is able to overfit it.
2024-06-10T14:46:38Z
Thanks, I will have a lookI tried lower values for both before going up. The higher values seemed to work a little better but still not good.I used a different approach, adding the SFTTrainer to the fine tuning process, which makes things work well.
2024-06-10T14:58:32Z
Yes see also my fine-tuning notebook regarding Mistral:Transformers-Tutorials/Mistral/Supervised_fine_tuning_(SFT)_of_an_LLM_using_Hugging_Face_tooling.ipynb at master · NielsRogge/Transformers-Tutorials · GitHub