How to view the changes in a model after training?
Hello,I trained a BART model (facebook-cnn) for summarization and compared summaries with a pretrained modelmodel_before_tuning_1 = AutoModelForSeq2SeqLM.from_pretrained(model_name)trainer = Seq2SeqTrainer(model=model,args=training_args,data_collator=data_collator,train_dataset=train_data,eval_dataset=validation_data,tokenizer=tokenizer,compute_metrics=compute_metrics,)trainer.train()Summaries from model() and model_before_tuning_1() are different but when i compare the model config and/or print(model) it gives exact same things for both.How to know, what exact parameters have this training changed?
2021-11-09T11:25:49Z
When fine-tuning a Transformer-based model, such as BART, all parameters of the model are updated. This means, any tensor that is present inmodel.parameters()can have updated values after fine-tuning.The configuration of a model (config.json) before and after fine-tuning can be identical. The configuration just defines basic hyperparameters such as the number of hidden layers, the number of attention heads, etc.
2021-11-09T11:34:06Z
so i should check model.parameters() for the difference? thanks
2021-11-09T11:38:04Z
Yes, indeed.
2021-11-10T12:40:42Z
I checked (model.parameters) and its coming exactly identical