Dataset confusion for distilroberta trained on squad2
Hi,As pertwmkn9/distilroberta-base-squad2 · Hugging Face, the exact and f1 scores achieved while eval are given on total for 6078 instances. The official dataset for SQUAD2 has 11873 instances (refer to official website).I searched for this subset for 6078 instances on google and got this -squad/data at master · elgeish/squad · GitHub- this has 6078 instances.But even with this subset, the exact and f1 scores I’m getting are around 58 and 62 only using the same run_squad.py script – while the reported numbers are 70 and 74.Need help if the model provided is properly trained or not and need clarity on the dev dataset being used.