Handling long text in BERT for Question Answering
I’ve read post which explains how the sliding window works but I cannot find any information on how it is actually implemented.From what I understand if the input are too long, sliding window can be used to process the text.Please correct me if I am wrong.Say I have a text"In June 2017 Kaggle announced that it passed 1 million registered users".Given somestrideandmax_len, the input can be split into chunks with over lapping words (not considering padding).In June 2017 Kaggle announced that # chunk 1
announced that it passed 1 million # chunk 2
1 million registered users # chunk 3If my questions were"when did Kaggle make the announcement"and"how many registered users"I can usechunk 1andchunk 3andnot usechunk 2at allin the model. Not quiet sure if I should still usechunk 2to train the modelSo the input will be:[CLS]when did Kaggle make the announcement[SEP]In June 2017 Kaggle announced that[SEP]and[CLS]how many registered users[SEP]1 million registered users[SEP]Then if I have a question with no answers do I feed it into the model with all chunks like and indicate the starting and ending index as-1? For example"can pigs fly?"[CLS]can pigs fly[SEP]In June 2017 Kaggle announced that[SEP][CLS]can pigs fly[SEP]announced that it passed 1 million[SEP][CLS]can pigs fly[SEP]1 million registered users[SEP]
2021-01-22T19:50:44Z
Hi@benj, Sylvain has a nice tutorial (link) on question answering that provides a lot of detail on how the sliding window approach is implemented.The short answer is that all chunks are used to train the model, which is why there is a fairly complex amount of post-processing required to combine everything back together.
2021-04-29T07:57:19Z
sir , I saw the article ,can you please tell after fine tuning the model , how to use it for question answering ?
2021-04-30T19:12:05Z
hey@rohit11, i think the simplest approach would be to load the fine-tuned model and tokenizer in a question-answering pipeline as follows:from transformers import pipeline, AutoTokenizer, AutoModelForQuestionAnswering

model_checkpoint = "/path/to/your/model"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
model = AutoModelForQuestionAnswering.from_pretrained(model_checkpoint)
pipeline('question-answering', model=model, tokenizer=tokenizer)you can consult thedocsfor more information on thepipeline
2021-08-13T16:33:52Z
HiWhen handling long context by breaking it into chunk where each chunk can be seprate example , is it likely that multiple chunks can have the answer independently or the answer can lie in an overlapping fashion spanning two chunks ?
2021-08-24T07:26:31Z
You would want to find the“best span”. Check Google’s implementation of Squadhere.
2022-01-19T14:37:54Z
@Sam2021As you can see in the notebook, they are finding answers in each chucks with modified start and end token. Also that’s why they are using offset tokens which maps tokens to original text.Answers do not get divided into two chucks that’s why they are using doc_stride which basically mitigate this problem
2022-03-10T09:24:19Z
Hello, I am having an issue with the same situation. I have long paragraphs from which i want to extract the answer. I split the paragraphs into chunks and used the (start_end) with the highest probabilities in each chunk (for each paragraph) . I saw the notebook and the solution of using the n_best (start_end) and check the score of all the combinations is interesting. But I wonder if there is a more elegant way to decide about which (start_end) token to choose among  the chunk.Is there an official implementation of this problem?the best span function implemented in the official code doesn’t really solve the problem of having start index in chunk 2 and the end index in chunk 9 for example. It just solves the problem of picking the righ  sub-paragraph.Thanks in advance.