Empty BERT Model, any help?
I am trying to train BERT For Masked LM from the BertForMaskedLM model.My question is:Is this model “empty”?Let me explain further:WithBertForMaskedLM.from_pretrained("bert-base-uncased")we load the “bert-base-uncased” model and can use it.But if I call the model like this.# INITIALIZE THE MODEL
model_config = BertConfig(
        vocab_size = vocab_size, 
        max_position_embeddings = max_length
      )

model = BertForMaskedLM( config=model_config )Am I actually loading BertForMaskedLM without using the pretrained model? Is this model “empty” as i can assume?
2024-01-05T09:21:47Z
bert1017×321 31.4 KBThe way you instantiate the model does not load the pretrained weights .So, you would be correct in saying it’s “empty”.
2024-01-12T10:32:33Z
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.