Can t5 be used to text-generation?
Hello to all,I’m following this tutorial:https://huggingface.co/blog/how-to-generatewhich says: " Auto-regressive language generation is now available forGPT2,XLNet,OpenAi-GPT,CTRL,TransfoXL,XLM,Bart,T5in both PyTorch and Tensorflow >= 2.0!"so I wanted to try to do the same, they just change the model to T5. However even though the model runs, the output is very strange. This is the code:!pip install transformers
import tensorflow as tf
from transformers import TFT5ForConditionalGeneration, T5Tokenizer

tokenizer = T5Tokenizer.from_pretrained("t5-small")
model = TFT5ForConditionalGeneration.from_pretrained("t5-small", pad_token_id=tokenizer.eos_token_id)
input_ids = tokenizer.encode('Hello, my dog is cute', return_tensors='tf')
greedy_output = model.generate(input_ids, max_length=50)
print("Output:\n" + 100 * '-')

print(tokenizer.decode(greedy_output[0], skip_special_tokens=True))I get this output:Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello HelloBecause of this, and taking into account that I have not found many text-generation examples with t5, I would like to ask if this is possible? if so, why my output is so strange?
2020-09-11T15:47:47Z
My mistake, reading the documentation is required that the first token is the task, for example ‘summarize’.
2021-05-25T12:52:32Z
Right, I know about the task preamble, but I was wondering as well what task to specify for plain old auto-regressive text generation?
2021-05-29T08:33:43Z
Yes, using summarize task works. Here is an example I used:from transformers import T5ForConditionalGeneration, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("t5-base")
model = T5ForConditionalGeneration.from_pretrained("t5-base")
input_ids = tokenizer.encode('summarize: I enjoy walking with my cute dog', return_tensors='pt')
greedy_output = model.generate(input_ids, num_beams=7, no_repeat_ngram_size=2, min_length=50, max_length=100)
print("Output:\n" + 100 * '-')

print(tokenizer.decode(greedy_output[0], skip_special_tokens=True))Output:I enjoy walking with my cute dog - he is a joy to walk with and is very affectionate with me. I like to spend time with him on walks with his kitty cat, leo, who is so cute!
2021-07-23T19:20:51Z
I want to fine-tune the model for my task. but when I trying to use “transformers/examples/pytorch/language-modeling/run_clm.py” gives this error.ValueError: Unrecognized configuration class <class 'transformers.models.t5.configuration_t5.T5Config'> for this kind of AutoModel: AutoModelForCausalLM.
Model type should be one of RoFormerConfig, BigBirdPegasusConfig, GPTNeoConfig, BigBirdConfig, CamembertConfig, XLMRobertaConfig, RobertaConfig, BertConfig, OpenAIGPTConfig, GPT2Config, TransfoXLConfig, XLNetConfig, XLMConfig, CTRLConfig, ReformerConfig, BertGenerationConfig, XLMProphetNetConfig, ProphetNetConfig, BartConfig, MBartConfig, PegasusConfig, MarianConfig, BlenderbotConfig, BlenderbotSmallConfig, MegatronBertConfig.Is there any solutions for this?
2021-07-26T13:09:31Z
As the error indicates, you cannot use therun_clmscript with a T5 model as there is no version of T5 with a causal LM head. The error listed all the possible architectures you can use instead.
2021-08-05T17:33:23Z
Hi@sgugger,I want toadd some small rules when generating the output text.For e.g:- First number should be larger than the second generating number in the generating sentence.Is that task is feasible inT5?
2023-04-26T06:40:03Z
sgugger:As the error indicates, you cannot use therun_clmscript with a T5 model as there is no version of T5 with a causal LM head. The error listed all the possible architectures you can use instead.hello!@sgugger, I’m reproducing a model for a paper and need to use t5-base model for Casual Language Modeling. I find nothing about it, would it be possilble to ask you?