-100 in predictions
I am using the HuggingFaceSeq2SeqTrainerwithpredict_with_generate=True.I get the value-100in my predictions during the validation step. This makes the tokenizer fail (and has absolutely no sense).Why?I can share the code, but, based on my experience, a long piece of code turns readers away more easily than a short question.
2024-12-20T07:34:14Z
Where can I read how and with which tokens the generated sequences are padded?I cannot find any information on this.These are the predictions I get (fromSeq2SeqTrainerwithpredict_with_generate=True) when I set max_new_tokens=100:preds[78]: [    0     3     2  3247  3321  3155     3     2  1018  6327  3155     3
     2  1018  6327  3155    37     3    75    23 17436    19     3     9
     3   729   302    13     3    75    23 17436     7    24   619    16
     8     3   729   302    96   254    23 17436   121     1  -100  -100
  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100
  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100
  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100
  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100
  -100  -100  -100  -100]The generated output is padded with thelabel_pad_id=-100instead of the tokenizer padding token. Why?When I use pure PyTorch code the predictions are padded with the token “0”.