Fine-tuning LLM for RAG
I am currently fine-tuning a LLM with a custom QA dataset, and was wondering whether there would be a significant difference whether I ran QLoRA fine-tuning with a model initialized withAutoModelForCausalLMand a model initialized withAutoModelForQuestionAnswering. If there does exist a significant difference, which of the two is preferrable?For additional context,  the dataset I will be fine-tuning with consists of three columns:question,context, andanswer. The three columns are then formatted with a prompt along the lines of### Instruction
Use the context below to generate an answer to the provided question. If the context does not contain sufficient information, state that an answer could not be found.

### Context
{context}

### Question
{question}

### Response
{response}
2024-06-10T07:14:00Z
Hi,The difference is generative vs. extractive question answering.TheAutoModelForQuestionAnsweringclass is meant for extractive question answering, i.e. the model is a classifier and needs to determine which text token of the context is at the start of the answer and which part of the context is at the end.TheAutoModelForCausalLMclasss is meant for generative question answering, i.e. the model is a generative model (like ChatGPT) and can generate any text given the context + question. This means that the model can give answers that go beyond the literate text of the context.
2024-06-18T00:08:03Z
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.