Missing keys "model.embeddings.position_ids" when loading model using state_dict
I have saved the model likemodel_state_dict = model.module.state_dict()torch.save({‘model_state_dict’: model_state_dict},  osp.join(save_dir, ‘best.ckpt’))Now I try to load the model likemodel_path = “./models/best.ckpt”ckpt = torch.load(model_path)model.load_state_dict(ckpt[‘model_state_dict’])then it has the error. There are “position_ids” in model, but not in the saved ckpt file.Anyway to skip loading position_ids ?
2020-08-19T17:29:39Z
Hi@jyliu, is there any specif reason for not using.save_pretrainedand.from_pretrained
2020-08-20T03:47:43Z
Thanks for your reply. Because the bert-model is a part of my whole model. I directly saved the whole model. So what is the best practice under this situation ?
2020-08-20T14:39:18Z
Will it be possible for you to create colab for this ?Also, to take advantage of.from_pretrainedand.save_pretrainedyou can sub-class theBertPretrainedModeland add the additional layers in it.  Seethesetask specific bert models, they use bert and additional layer on top of it and subclassBertPreTrainedModel.let me know if this solves your problem.
2020-08-21T04:02:42Z
Thanks. This will work for me.