Fine tuned phi2 model loses context once loaded from local
Hi Awesome people,Currently I am exploring the phi-2 fine tuning and I am at the phase where results it yielded is pretty good. However, I am facing an issue when I try to load and run the model for inference. It stared giving hallucinations and non-relevant answers. Some times similar questions as the answers to the question prompted.I have saved the model after finetuning (using PEFT) withpipelineand also withSFTTrainer. i.e.(at separate instances)pipe.save_pretrained('./path_to_local_dir')

trainer.save_model('./path_to_local_dir_2')and tried to load them viaAutoModelForCausalLM.from_pretrained('./path_to_local_dir')

pipe = pipeline(task="text-generation", model='./path_to_local_dir', tokenizer=tokenizer, max_length=250, trust_remote_code=True)but in both cases, my finetuned LLM (phi-2) is behaving abnormally. Could anyone please suggest where I have gone wrong ? Every help is appreciated.Thanks