T5 Gen Len is only 1/14 of max_target_length
Following thisHuggingFace Google Colab Page, I fine-tuned t5-small for text summary.Did some inferencing with some examples and it does some good job creating one sentence summary.But the problem is Gen Len.While training I set themax_target_lengthto be 128. And also average number of words of summary in dataset is 54.But Gen Len is only being 18. Can anyone know what would be the cause of this?ThanksHere’s full training result:EpochTraining LossValidation LossRouge1Rouge2RougelRougelsumGen Len10.6922000.64036935.38150026.48010033.18820034.24660018.97680020.5763000.61385735.90300027.49400033.93210034.90480018.91250030.4517000.58789137.06790028.70070035.13240036.06440018.91070040.3523000.58781637.16860029.10270035.18360036.26100018.912500
2023-07-06T18:46:26Z
I’m seeing something similar when trying to train a BART model…It only produces predictions of 20 tokens which seems to be the default for the generate function, but no matter what config parameter i set (max_length, max_token_length etc.) during the compute_metrics stage of a Trainer loop, it only produces predictions of 20 tokens…
2023-07-06T19:27:00Z
I found the reason for my problem. During finetuning I only produce labels of 20 tokens (which is some sort of default I assume), so that is what the model learns…Only produce summaries of 20 tokens long, and it was caused by my preprocessing tokenization step of my training data.In my tokenization step for the labels during finetuning, I used this as my tokenization step:tokenized_targets = self.tokenizer(
                text_target=batch[self.label_column],
                max_length=self.model.config.max_token_length,
                **self.tokenizer_config,
                **kwargs,
            )But when I change it to this, it does produce longer labels than 20 tokens. Not sure why I need to provide thebatch[self.label_column]as an input to the tokenizer, effectively twice, but this way it does “listen” to the max_length (which I set in the model.config in my class to 128)…tokenized_targets = self.tokenizer(
                batch[self.label_column], # Added this to make it work
                text_target=batch[self.label_column],
                max_length=self.model.config.max_token_length,
                **self.tokenizer_config,
                **kwargs,
            )Hope that helps you since your generated lenght is awefully close to 20 and there may be some that are shorter, hence your avarage lenght < 20…Also, not sure where your argumentmax_target_lengthis set, butmax_length(or, for V5 version it will bemax_token_length) is the correct argument for the tokenizer, so you may also have to look at that.
2023-10-05T07:19:02Z
I have found the solution to this. It is because of thegeneration_max_lengthparameter in the training args.generation_max_length(int,optional) — Themax_lengthto use on each evaluation loop whenpredict_with_generate=True. Will default to themax_lengthvalue of the model configuration.Please refer this section:TrainerThere search for"generation_max_length".There also isGenerationConfigthat you can use. But thisgeneration_max_lengthwill solve the issue