Failed to train bart-cnn from bart-base using my own code
Update: problem solved. caused by a bug in model saving functionThere’re not much difference on Rouge score before and after training.The generated summarization of same article is almost same.Need some suggestion.I used original setting infairseq.label_smoothed_cross_entropy,label-smoothing0.1,attention-dropout0.1,weight-decay0.01,lr-schedulerpolynomial_decay,adam-betas“(0.9, 0.999)”adam-eps1e-08,LR=3e-05some difference:batch-size 3,gradient_accumulation_steps 16,adamW,fp32,clip-norm 1.0,trained for 7 hours on 2 tesla P4.Loss (my fault made it hard to read):image1475×463 36.5 KBModel seems not converge .Rouge score before training:“rouge1”: 37.325248080026086,“rouge2”: 18.03751262341448,“rougeL”: 25.82438158757282Rouge score after training:“rouge1”: 37.818,“rouge2”: 17.246,“rougeL”: 23.7038Update: problem solved. caused by a bug in model saving function
2020-10-07T16:15:12Z
AFAIK there isn’t bart-base checkpoint trained for summarization, how did you get those metrics without training ?
2020-10-08T01:55:30Z
I directly calculate rouge score on cnn/mail before training.
2020-10-08T01:56:26Z
I think there’re bugs in my code, but have no idea what kind of bug would cause the problem.