Error: The model weights are not tied. Please use the `tie_weights` method before using the `infer_auto_device` function, even after adding model.tie_weights()
Hi,I am trying to use ‘Accelerate’ for running inference on aPMC-Llama modelusing a GPU and CPU.Here is my code,import transformers
from accelerate import init_empty_weights
from transformers import AutoConfig, AutoModelForCausalLM

config = AutoConfig.from_pretrained("chaoyi-wu/PMC_LLAMA_7B")
with init_empty_weights():
    model = AutoModelForCausalLM.from_config(config)

from accelerate import infer_auto_device_map, init_empty_weights
model.tie_weights()
device_map = infer_auto_device_map(model)
# Getting the warning: The model weights are not tied. Please use the `tie_weights` method before using the `infer_auto_device` function.I am not able to understand why I am getting this error even though I have added the statement, ‘model.tie_weights()’Thank you. Any help would be highly appreciated.
2023-07-14T03:58:36Z
I had this same issue, if anything could please answer the OP question that would be super helpful!
2023-07-14T13:31:46Z
@abhibhaupgrading to transformers >= 4.28.1!pip install transformers==4.28.1
2023-07-14T21:13:11Z
@FalahThanks! I am already using transformers version -  4.30.2 . And still am getting the error.
2023-07-15T06:14:25Z
@abhibhau can resolve it in google colabcolab.research.google.comGoogle Colaboratory
2023-07-22T17:20:49Z
Thanks@Falah, I tried running my code without the ‘model.tie_weights()’ line. It still doesn’t give the warning. I tried running on google colab. Should I ignore the warning?
2023-07-23T10:04:57Z
@abhibhaIf you are sure that the warning was triggered by the presence ofmodel.tie_weights(), and the warning disappears when that line is removed, then it’s likely that the warning is related to the usage ofmodel.tie_weights()in your specific context.Without knowing the specific code and the warning message you received, it’s challenging to determine the exact cause of the warning and whether it is safe to ignore. However, I can provide some general guidance:Understand the purpose ofmodel.tie_weights():Check the documentation and version compatibility:Make sure to refer to the documentation of the library or framework you are using formodel.tie_weights().Consider the impact on your specific use case:Depending on your specific use case, tying weights may or may not be necessary. If you are achieving the desired results without usingmodel.tie_weights(), and the warning disappears after removing it, it might be acceptable to leave it out. However, it’s crucial to consider the implications on your model’s performance and training process.Test and validate your results:If you decide to proceed without usingmodel.tie_weights(), thoroughly test your model and validate the results to ensure that it performs as expected and achieves the desired outcomes.
2024-02-20T08:38:18Z
Why post answers from chatGPT?This answer clearly contains no crucial information and it is not the solution to this error.
2024-02-20T14:02:29Z
@NitzanBarAI Models are the only solution to all programming problems for us as system developers. because traditional methods are slow, boring, and annoying, and we do not reach results in a short timeIn your opinion, what is the solution to the problem!!! It was solved in Google Colab previously…I need a solution from you, please benefit from and share it