Is Eval and Validation same in Trainer API?
Hi, I am a bit confused if the eval dataset parameter is used during the training.#Trainer itself.
trainer = Trainer(
    model,
    args,
    train_dataset=tokenized_datasets_train,
    eval_dataset=tokenized_datasets_val,
    tokenizer=tokenizer,
    compute_metrics=compute_metrics,
    data_collator = data_collator_
)is the eval_dataset only used when we do trainer.evaluate() ?
2021-09-14T00:42:50Z
Yes, it’s the default dataset used for that method (which will be used if you pass aneval_strategyto evaluate every epoch or n steps).
2021-09-14T08:16:27Z
Okay perfect, then i will not put my test set over there Thanks!@sguggerAlthough i want to get the f1 score of my model on the test set, do you know if there is a metric api from hugginface i could use?If so could you please link me a small script?
2021-09-14T12:12:03Z
You can look at any of theexamplesor thecourse section on the Traineron how to du this using acompute_metricsfunction.
2021-09-14T12:36:43Z
Huge thanks again!Have a nice day