T5forConditionalGeneration
Hello,I am a newbie in using transformers and never used it before.I want to know, what’s the difference between T5Model and T5forConditionalGeneration? Where are they used?
2020-09-15T07:59:31Z
Hi@ashiishkarhadeT5Modelcontains theencoder(stack of encoder layers) anddecoder(stack of decoder layers)  without any task specific heads. It returns the raw hidden states of the decoder as output.T5ForConditionalGenerationalso contains theencoderanddecoderand adds an additional linear layer (lm_head) which takes the final hidden states ofdecoderand generates the next token.For fine-tuning the model for seq2seq generation you should useT5ForConditionalGeneration, if you want to add some different task specific head then  you canT5Model.And almost all library models have this structure, a base model which returns raw hidden states and additional models with task specific heads(ForSequenceClassification,ForQuestionAnsweringetc) on top of the base model.
2020-09-15T08:31:56Z
Thank you for the brief .