Difference between using or not compute_metric
Hello, I am newbie in HF and I found there are two different implementations as follows:def compute_metrics(eval_pred):logits, labels = eval_predpredictions = np.argmax(logits, axis=-1)return metric.compute(predictions=predictions, references=labels)In train API, some codes call compute_metrics while some does not. What is the differnce between using and no using this metrics function. Can you please explain how it can effect the system predictions??trainer = Trainer(model=model,args=training_args,train_dataset=dataset_train,eval_dataset=dataset_test,#compute_metrics=compute_metrics,callbacks=[EarlyStoppingCallback(early_stopping_patience=1)])
2023-11-27T08:48:17Z
Compute metrics is to help track how the training is performing. It does not affect the training itself.So think of it is a reporting function that calculates extended metrics to help you evaluate the quality of training. You can then use the metrics it reports to fine tune the training.see this example here.huggingface.coFine-tuning a model with the Trainer API - Hugging Face NLP CourseWe’re on a journey to advance and democratize artificial intelligence through open source and open science.
2023-11-27T09:33:55Z
thank you so much