How to load weights which was trained older version of PyTorch?
Hi,I trained a RoBERTa model on PyTorch 1.6 and tried to load the weights on different server which had PyTorch 1.4.But I could not load the weights and got an error which said,OSError: Unable to load weights from pytorch checkpoint file. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.I can upgrade PyTorch which is not a problem but I’m curious how can I load weights trained on older version of PyTorch?Environments of both servers:ubuntu: 18.04 ( both servers )transformers: 3.1.0 ( both servers )Python 3.7.9 and Python 3.7.3Thanks in advance.
2020-09-27T13:13:27Z
AFAIK PyTorch versions shouldn’t make a difference. Check your path, check that the file is not empty. Also post the code that you use to save and load the model.
2020-09-27T23:31:18Z
Thank you for your response.I used the example codehere.But dataset is not Esperanto.Anyway I’ll check my codes more.It seems something happened while reading weights file because of os or python version difference…
2020-09-29T06:42:00Z
using torch.save(model.state_dict(), os.path.join(output_dir, ‘pytorch_model.bin’), _use_new_zipfile_serialization=False) instead of model.save_pretrained() or trainer.save_model()
2020-09-29T07:26:55Z
It does with PyTorch 1.6 as nowtorch.saveuses a new zip file-based file format (unless you pass a _use_new_zipfile_serialization flag as False). Of course, older PyTorch versions can’t load this format; I run into this issue often.
2020-09-29T09:24:54Z
Wow, I didn’t know about this, thanks@BinhMinhs10and@vblagoje!
2021-04-21T07:05:20Z
Thank you!!! I updated my torch1.2 to 1.8 and solved the problem!!!
2021-05-12T07:03:17Z
I encounted this error too when I used pytorch 1.4. But when I use pytorch 1.8, this error dispeared
2022-02-10T02:02:59Z
Any way to overcome this with a PyTorch version below 1.6? I’m using 1.5 on my system and can’t upgrade anytime soon! :X