Is it possible to use a pre-trained Bert model with a modified type_vocab_size parameter?
Hello,Is it possible to load a pretrained Bert model but with a differenttype_vocab_sizevalue?For exmaple.:The pretrained model:stsb-bert-baseTheconfig.jsonof thestsb-bert-basemodel:Now, I want to set thetype_vocab_sizeto 20, is there a possible way that I can still use the pretrainedstsb-bert-basemodel as a start point to fine-tune on my own dataset? Or I have to train the model from scratch?As a reference, I tried to loading the pretrained model with a modifiedtype_vocab_size=20, but got an error as below:“RuntimeError: Error(s) in loading state_dict for BertModel: size mismatch for bert.embeddings.token_type_embeddings.weight: copying a param with shape torch.Size([2, 768]) from checkpoint, the shape in current model is torch.Size([20, 768]).”Thank you!Br,Na