Use Pretrained T5 for Summarization
Hello,Is there any code snippet of how to use T5 pretrained model in order to do summarization?
2020-11-11T15:40:36Z
I used the following code to do my task:from transformers import T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained('t5-small')

model = T5ForConditionalGeneration.from_pretrained('t5-small', return_dict=True)

input = "This  is a summarization example. This is a large  sentence."
input_ids = tokenizer("summarize: "+input, return_tensors="pt").input_ids  # Batch size 1

outputs = model.generate(input_ids)

decoded = tokenizer.decode(outputs[0], skip_special_tokens=True)

print(decoded)
2020-11-12T12:57:11Z
You can take a look at the contributed notebooks here:https://github.com/huggingface/transformers/tree/master/notebooks#community-notebooksFor instance this one is about finetuning (and using) T5 for summarization:https://github.com/abhimishra91/transformers-tutorials/blob/master/transformers_summarization_wandb.ipynb
2021-07-02T20:37:59Z
Hey there@thomwolf, I happened to observe that the ROUGE scores achieved on the tutorial notebook that you have posted are kind of far from the results presented on the paper (roughly 43 for rouge-1).Are there any fine-tune implementations that have results closer to the original paper?Thanks a lot in advance!