Loading pretrained weights into model for sequence classifcation
Hi there,I am a bit confused as to how saving models works. I have been pretraining with an ElectraForPreTraining model and have saved it using save_pretrained(). However, I am now interested in fine-tuning the model on a sequence classification task, so I figure I would have to load my pretrained weights into an ElectraForSequenceClassification model, except the ElectraForPreTraining model has a different head on the discriminator than the ElectraForSequenceClassification and I am not sure how to handle that correctly. Does the from_pretrained() method handle this out of the box and if not what would I have to do?
2020-07-22T04:44:00Z
Hi@rsvarmathe.from_pretrainedmethod handles this.  You can doElectraForSequenceClassification.from_pretrained("path to your pre-trained electra model")
2020-07-22T07:56:31Z
Hi@valhalla,Thanks, thats what I needed to know!