How to Avoid Overfitting?
I have a dataset with 7,000 lines I plan to train with GPT2.  I am unsure how many steps I should train it to, however. Does anybody have any advice on how I can avoid overfitting?
2020-07-30T07:08:11Z
Avoiding overfit is a big question, I’ll just say some terms I’ve know / heard may help.data augmentation, layer wise lr, weight decay, gradient clipThere are lots of things you can explore.