Converting LLaMa 2 bin files to safetensors changes the output
Hi,I fine-tuned LLaMa 2 to test its query classification quality and after I saved my final model, I converted thebinfiles tosafetensorsand used it in TGI. But I noticed I am getting completely different results compared to just usingAutoModelForCausalLM.from_pretrained()andmodel.generate(). Note that all other parameters like top_p, top_k, etc. are the same and temperature is set to a small positive value of 0.01. After a lot of testing, I am confident thatsafetensorsare the only variable between the two.Is this a known fact or a bug in conversion? I used the following code to convert:huggingface.coconvert.py · safetensors/convert at mainWe’re on a journey to advance and democratize artificial intelligence through open source and open science.Thanks,Milad