Training megatron as lm in QAModel with squadv2 dataset
No, you should be able to get much higher accuracies for 'megatron-bert-345m-uncased' on Squadv2 dataset.
This what I got after 2 epochs of training em=81.35%, f1=84.75%.
I used batch size=12, precision=16, amp_level=01 Actually, I think I know the reason for the low numbers. We had a recent bug in QA model's code which was just corrected. Please take the latest update from Nemo 'main' branch and it should train properly for you. @avikpal00 could you share your omegaconfig of your settings that is printed at the beginning of execution if the suggestions by @vladgets do not solve your problem? @avikpal00 closing due to inactivity 
Hello,
On training QAModel with squadv2 dataset keeping pretrained_model_name=megatron-bert-345m-uncased and version_2_with_negative=True does not seem to train properly (val_em=50.07, val_f1=50.07). But this is not the case when I keep pretrained_model_name=bert-base-uncased or albert-xlarge-v2, in both of which I get good validation results (bert em=72, f1=76 ; albert-xlarge em=82.8 f1=86.2). This tells that megatron is not handling questions with is_impossible flag. Do I need to configure something else for megatron for it to work with squadv2 structure?
