Help with merging LoRA weights back into base model :-)
As best as I can tell, the LoraModelmerge_and_unloadattribute (peft/lora.py at main · huggingface/peft · GitHub) merges LoRA weights back into the main model.However, I am having trouble getting a LoraModel type from my PeftModelForCausalLM. My current workflow is to define a pretrained model, define a LoraConfig, and use theget_peft_modelfunction to being training. This works great, but I want to be able to merge the weights back into the base model and save.My working assumption is that I need to either convert my PeftModelForCausalLM into a LoraModel or initialize the model as a LoraModel prior to training. However, when I copy the example in the LoraModel docstring (peft/lora.py at main · huggingface/peft · GitHub), I get anTypeError(TypeError: LoraModel.init() missing 1 required positional argument: ‘adapter_name’). When I try passing a “lora” as a adapter name, I get another error.I think that I am fundamentally thinking about this in the wrong way and would love some pointers. Both Google and Copilot chat have not been able to solve my problem.
2023-05-25T22:10:50Z
I figured this out. The solution is quite simple.A PeftModelForCausalLM actually inherits the LoraModel methods, so you can callmerged_model = merged.merge_and_unload()to get back a base model with the LoRA weights applied.My IDE would not autocompletemerge_and_upload, so I assumed the method wasn’t available. I still don’t need in the code where this method is inherited and would love for someone to point this out to me if feeling charitable.
2023-06-20T16:45:06Z
Looking to get a model file from a base + lora myself. Can you explain in more detail how you were able to do it?
2023-06-22T01:48:56Z
Try this. Basic steps are to:1/ load the base model2/ train the base model3/ save the LoRA adapter4/ reload the base model at half/full precision5/ merge the LoRA weights with the base model6/ savebase_model = AutoModelForCausalLM.from_pretrained(“base_model”, load_in_8bit=True, torch_dtype=torch.float16, device_map=“auto”)base_model = prepare_model_for_int8_training(base_model)peft_model = get_peft_model(base_model, peft_config)training_args = TrainingArguments()trainer = Trainer()trainer.train()peft_model.save_pretrained(lora_adapter, save_adapter=True, save_config=True)model_to_merge = PeftModel.from_pretrained(AutoModelForCausalLM.from_pretrained(base_model).to(“cuda”), lora_adapter)merged_model = model_to_merge.merge_and_unload()merged_model.save_pretrained(merged_model)
2023-07-22T07:42:46Z
These are the correct steps to create a model version form a base model and my train?
2023-09-15T13:17:39Z
A related question, why the model size in disk is almost doubling after merging even when the number of parameters remaining the same?
2024-01-21T05:32:31Z
But when I try to save the model weights on model_to_merge then I still get only the adapter safetensors and not the safetensors for the entire model.How do i get that?
2024-01-31T09:25:53Z
I have the same question, why the size difference in the base model and merged model?
2024-04-29T14:48:48Z
I see 2 potential reasons for that, unsure if any of them is applicable to your use case as I cannot see your code.The merged model (base model + LoRA adapters) has the numer of parameters of the base model + the number of parameters of the inserted LoRA adapters. For this reason,# params merged_model > # params base_model, therefore the increase in sizeWhen reloading the model, make sure to provide the same data type as in training to ensure the same size is mantained. For example, if the trained model was loaded in half precision, also the model_to_merge should be loaded in half precision to enforce comparable size e.g:model_to_train = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16) 
...
model_to_merge = PeftModel.from_pretrained(AutoModelForCausalLM.from_pretrained(base_model, torch_dtype=torch.bfloat16), lora_adapter)In the example provided by@dmckinno, themodel_to_trainis loaded in 8 bits (load_in_8bit=True), and thenmodel_to_mergeis loaded in full precision, since no parameter is provided and the default will be used (float32)
2024-05-01T14:33:04Z
Hi,I explain it in more detail :Config.json is not saving after finetuning Llama 2 - #6 by nielsr, hope you find it useful!And this is a bit related:Further finetuning a LoRA finetuned CausalLM Model - #4 by nielsr
2024-06-20T08:13:40Z
Wasn’t here missing the part where you can push this into HUB?merged_model.push_to_hub("repo_id")
tokenizer.push_to_hub("repo_id")