Combine multiple metrics in compute_metrics for validation
HiI am trying to get precision / recall / accuracy scores computed during the evaluation of the model during training. My compute_metrics() function is as follow:metric1 = evaluate.load("accuracy")
metric2 = evaluate.load("precision")
metric3 = evaluate.load("recall")
def compute_metrics(eval_pred):
    predictions, labels = eval_pred
    predictions = np.argmax(predictions, axis=1)
    #
    accuracy =  metric1.compute(predictions=predictions, references=labels)["accuracy"]
    precision = metric2.compute(predictions=predictions, references=labels,
                                average="micro")["precision"]
    recall = metric3.compute(predictions=predictions, references=labels,
                                average="micro")["recall"]
    #
    return {"precision": precision, "recall": recall, "accuracy": accuracy}TheTrainingArgumentsandTrainerare as follows :training_args = TrainingArguments(
    output_dir=str(local_model_pth / model_name),
    learning_rate=2e-5,
    per_device_train_batch_size=8,
    per_device_eval_batch_size=8,
    num_train_epochs=1,
    weight_decay=0.01,
    evaluation_strategy="steps",
    eval_steps=200,
    save_steps=20000,
    save_strategy="steps",
    load_best_model_at_end=True,
    push_to_hub=False,
)
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_ds["train"],
    eval_dataset=tokenized_ds["test"],
    tokenizer=tokenizer,
    data_collator=data_collator,
    compute_metrics=compute_metrics,
)

trainer.train()Every thing seems to run fine, but the output I get when the model is training is as follows :The precision / recall / accuracy have the same value . I have been through the help here, and Internet / HuggingChat without any working solution.Any idea ?
2024-06-04T07:47:40Z
Theaverage="micro"seems to be a no go (which makes sense, which of the N classes would you display). Discovered alsoevaluate.combinein the Evalute library docsThe implementeametric = evaluate.combine(["precision", "recall"])
def compute_metrics(eval_pred):
    predictions, labels = eval_pred
    predictions = np.argmax(predictions, axis=1)
    #
    return metric.compute(predictions=predictions, references=labels, average="macro")This implementation will not allow to combineaccuracywithprecision/recallin the case of a multi-class use case. Indeed, if you combineaccuracywithprecision/recall, you cannot useaverageas an argument inmetric.compute(accuracy does not expect it). And if you leave it as default,precision/recallis set to binary as default.