Training los 0.0 right from the start
Hi,I’m trying to train distilgpt2 following the given tutorial:huggingface.coCausal language modelingWe’re on a journey to advance and democratize artificial intelligence through open source and open science.I have more or less the same code in my notebook, except the dataset I am using is this one:kaggle.comDonald Trump Rally SpeechesDive into 35 rally speeches by Donald Trump.For some reason the training loss seems to be 0.0 right from the start. I don’t really know why.