Combine base model with my Peft adapters to generate new model
Hi All,I am trying to merge my fine-tuned adapters to the base model. With thistorch.cuda.empty_cache()
del model
pre_trained_model_checkpoint = "databricks/dolly-v2-3b"
trained_model_chekpoint_output_folder = "/content/gdrive/MyDrive/AI/Adapters/myAdapter-dolly-v2-3b/"

base_model = AutoModelForCausalLM.from_pretrained(pre_trained_model_checkpoint,
                                  trust_remote_code=True,
                                  device_map="auto"
                                  )
model_to_merge = PeftModel.from_pretrained(base_model,trained_model_chekpoint_output_folder)
del base_model
torch.cuda.empty_cache()

merged_model = model_to_merge.merge_and_unload()

tokenizer = AutoTokenizer.from_pretrained(trained_model_chekpoint_output_folder)Thenmerged_model.save_pretrained('path')The generated model size is the aprox the double. (5.6Gb to 11Gb) My fine tunning basically add info about 200 examples dataset in Alpaca format.what am I doing wrong?
2023-09-15T13:15:55Z
Is this problem solved? I am having the same issue with Falcon 1b. The number of parameters remaining the same but the model size on disk is doubling from 2.4g to about 4.5g.