Modifying architecture of the models provided by the library
Are the models provided here easily modifiable to implement custom changes to the architecture? I plan to: pretrain a GPT model on my native language → add/modify layers but keep trained parameters - > finetune the model.I am adding/modifying intermediate layers and not adding layers after/before the model.Is it fine to modify the code and it will not break anything?
2023-03-13T11:06:58Z
Hey man, did you find a solution to this? I want to load the pretrained model weights and then add a layer after the encoder of the T5 model for translation task. I’m not sure how to edit the architecture of the given models.Pls share if you did!