Passing additional tensors into metric evaluation function
Hi,I was wondering if it is possible to pass additional labels into metric evaluation function. In more details, I want to calculate metrics using not all predicted tokens but only a small subset, and to do this I need to pass a tensor of indices of that tokens into the metric evaluation function.  But I don’t need this tensor for anything else, e.g. for calculating the loss.I tried to add this tensor into evaluation dataset, usinglabel_namesandremove_unused_columns = Falseoptions. But in this case the model (T5 encoder-decoder) sees unrecognized keywords and fails. Defaultremove_unused_columns = Truejust deletes the tensor from the batch.I am usingTransfomers version 4.7.0Any help is much appreciated! Thanks.
2021-07-26T13:07:09Z
If you leave it in your dataset, you will have an error since your model can’t consume it. You will need to write a manual evaluation loop for this use case.
2021-07-26T16:43:59Z
Got it. Thank you for the answer.