GPT2 finetuned with eos token will never yield eos token during generation
Hi,I finetune the smallest version of gpt2 (distilgpt2) trained on a dataset. The dataset consists only of texts and after some texts, an EOS token is inserted.Training is running decently, the loss is constantly decreasing.But using model.generate(input_ids, …) no matter what the model will always output tokens till the max_length has been reached.I think that the probability for EOS token has not been adjusted in the model well enough.Any tips to improve it or make the model generate EOS after some texts?
2022-05-31T07:14:09Z
I have encountered the same problem. Have you found any solutions?
2022-06-02T13:05:56Z
Finally, I found the reason.The DataCollatorForLanguageModeling always masks the pad_token in the labels and I set the pad_token = eos_token.
2023-05-26T09:17:44Z
Hi, I am encountering the same problem. How did you resolve this? this you change the pad_token to something else?
2023-05-28T01:37:24Z
I have the same problem, the model does not shut up…
2023-08-05T02:15:38Z
Medium – 26 Jul 23Solving The Issue of Falcon Text Generation Never StoppingHow to make an overly chatty bird stop talking.Reading time: 3 min readI believe the most elegant solution may be to switch to using the Seq2Seq DataCollator as described here, otherwise you can introduce a new padding token.
2024-04-12T09:10:12Z
I faced the same problem.The fine-tuning of Gemma 2 works well according to the loss functions.But after training the prediction was just eos eos.The solution in my case was simple:Set eos_token to Falsemodel = AutoModelForCausalLM.from_pretrained(model_id,tokenizer = AutoTokenizer.from_pretrained(model_id, add_eos_token=False)