Why `from_pretrained` method still works when model config is mismatched?
Hi,This might be a silly question. But I try to config a customized Bart, and usefrom_pretrainedmethod to load weights. And I expect some errors raised, as my config neither completely matches the config inconfig.json · facebook/bart-base at mainnorconfig.json · facebook/bart-large at main. But it didn’t.from transformers import BartForConditionalGeneration, BartConfig
myconfig = BartConfig(d_model=1024, max_position_embeddings=256, 
    encoder_attention_heads=8, decoder_attention_heads=8,
    encoder_layers=10, decoder_layers=10)

model = BartForConditionalGeneration(myconfig)
model = model.from_pretrained('facebook/bart-base')
model = BartForConditionalGeneration(myconfig)
model = model.from_pretrained('facebook/bart-large')Just wonder why…
2021-01-26T07:05:49Z
callingfrom_pretrainedinitializes a new model using the new config and returns it, which is why there is no error