How to get the loss history when use trainer.train()?
I am using the huggingface transformers. I noticed that when I call the train(), I can get a table contains the evaluation loss and training loss, how can I get the data in this table and use them to plot figures? (without wandb)Screenshot from 2022-05-04 01-36-421207×784 75.6 KB
2022-05-03T18:11:15Z
You should find them inTrainer.state.log_history. There is aparse_log_historyfunction in the modelcard module that might be helpful to clean those logs.
2022-05-04T09:59:17Z
Thank you !! I got it !!!
2023-04-30T00:27:07Z
Can you please share how to do it?
2023-04-30T20:20:45Z
Suppose you initialized:trainer = transformers.Trainer(...)Then you can access and convert the losses to dataframe by runningimport pandas as pd
pd.DataFrame(trainer.state.log_history)