Fine-tuning T5 for sentiment classification
Hi all,I would like to fine-tune a T5 model for sequence classification (specifically sentiment classification). However, all the tutorials are doing seq-2-seq analysis, such as text summarization as below. I understand why it uses ROUGE score for the cost calculation and it uses AutoModelForSeq2SeqLM package since it is seq-2-seq task. I think I need to change my cost function and also need to replace AutoModelForSeq2SeqLM by something? Any suggestions/ideas how to proceed?colab.research.google.comGoogle ColaboratoryThank youBest
2023-06-29T14:29:06Z
Hello@dammy,You can check outthis notebookto fine tune T5 on sequence classification (not in the classification way like encoder only models do but rather with conditional generation. In case you haven’t checked, I’d suggest you to use encoder only models with classification heads and see how they perform. It’s a simpler approach that would work better for classification IMHO.
2023-06-30T14:25:54Z
Thanks@merve, I have tried BERT as an encoder only method, the accuracy is fine. However, I want to see if I can have something better with T5 fine-tuned.
2023-12-22T23:23:12Z
@dammyhow did your experiment with T5 fine-tuning  on sentiment classification?