T5 fine-tuning for custom output
Hi,I am trying to fine-tune a model to extract custom data from a sentence. The data looks like this:texttargetWe are open from 8 am to 6 pm8-6The store working hours are 8-88-8Some random textnoneI tried to fine-tune the T5 base to do this task but the generated text didn’t follow the custom dataset.Steps I followed:Load T5ForConditionalGeneration and T5Tokenizer.Load my custom dataset.Tokenize the text data with a prefix for tokenizer “extract working hours:”.Train using Trainer.Save the model locally.Load model.Apply transformers pipeline to generate text.I also tried to use a custom function to tokenize the input and decode output instead of using the pipeline, which didn’t generate the expected output.Next, I copiedFine Tuning T5 Transformer Model with PyTorchcolab notebook and replaced the dataset with mine. The generated text wasn’t correct either.Is T5 the right model for what I am trying to achieve? If not, which model would you recommend for this task?Thank you!