Using SciBERT pretrained model to build a question answering model using scientific research papers as dataset
I want to build a model that is able to read a scientific research paper and answer questions based on that knowledge. I have been surfing the web and came across such a project but it used Longformer model which might have derailed its performance.I have also read that Scibert limits the tokens , therefore long text cannot be processed.my questions are :is it advisable to use scibert as the pretrained model for this task , if not which better model is out there ?will scibert work on long texts ?