【Solved】How can I get loss by using trainer when training gpt2?
I want to use trainer to train my gpt-2 model, while when I pass the dataset to the trainer, it tell me that there is a keyError: ‘loss’. So I try to find the cause and find that the function ‘compute_loss’ is not work as what I want. So I write a new compute_loss function but find that the inputs don’t have the column ‘labels’, which I create when using tokenizer, I think is may be deleted when used by trainer.I wonder how should I use trainer correctly so that I can make compute_loss work wellthis is my compute_loss function, but not work because it does not find a column named labelsimage922×297 20.4 KB
2022-07-20T14:53:22Z
there are more pictures about the problemthis is my dataset, it is only used to test how to use trainer so I don’t distinguish themimage1077×540 36.4 KB
2022-07-20T14:53:54Z
this is my trainerimage1188×757 41 KB
2022-07-21T08:52:24Z
By using GPT2LMHeadModel instead of GPT2Model, the problem doesn’t appear again. So I wonder what is the difference between them.