Question-Answering/Text-generation/Summarizing: Fine-tune on multiple answers
Hi all!Looking to fine-tune a model for QA/Text-Generation (not sure how to frame this) and I’m wondering how to best prepare the dataset in a way that I can feed multiple answers to the same question?My goal is to facilitate the creation of auniqueanswer to a given question that is based on the input answers. The answers are longer-form (2 to 3 sentences) and I want the model to output this kind of length too.For now my idea is to fine-tune GPT-2 and look at this as a text generation problem (but I don’t know how GPT-2 would treat multiple answers to the same question - would it adjust the weight simply in favor of the used tokens in the answers?). Maybe creating a summary of the given answers would have the same effect.What would be the best approach to this?
2020-12-16T01:18:57Z
Maybe@valhallahas some pointers?
2020-12-16T05:56:24Z
hi@enrico!I can think of two approachestext generation/text2textHear as you said you could feed the question and multiple answers to the model and ask it to generate the correct answer.One way to prepare the dataset for this is to use some delimiter tokens to separate the answers. For ex.Input text could becontext: context text question: question_text Answer1: answer1_text Answer2: answer2_textand output text would be the answer text.You could also delimit the answers usingseptokens if the models have them.Frame this as a multiple-choice problemHere instead of generating correct answer text, you ask the model to identify the correct answer within given answers discriminatively.You can use the*ForMultipleChoicemodels classes for this.You can find multiple choice example script inexamples/multiple-choicedir,here
2020-12-17T14:11:50Z
Thank you@valhallaleaning towards the first idea. Will give this a try
2020-12-18T09:04:42Z
Maybe you could try also this solutionhttps://huggingface.co/qa/
2020-12-18T09:17:55Z
@cahyathanks a lot! Looks really interesting didn’t know about it!
2020-12-18T10:02:17Z
@enricoplease have a look also on the discussion we had about itDemo of Open Domain Long Form Question Answering
2021-09-28T09:57:42Z
Hello can i use multiple context in a question and answer huggingface api?? I am actually trying to make a context that involves several paragraphs about all the past 46 US presidents and I cannot seem to put all those under a single context. Can you please help me ???
2021-11-20T19:03:26Z
Hi there,If I understood your situation, you have a context, questions and answers, but your context is too big to feed the model in one passage.I suggest you to take a look onHugging Face’s question answering example notebook. They manage to solve this problem splitting up the context in several parts, when necessary. During training, if the context split does not contain the answer, they point the answer to the CLS token. Finally, during evaluation, they search for the answer in all splits and consider the most likely answer.