Why different num_train_epochs give different results?
I am running the sample code [Text classification](https://Textclassification)When I set num_train_epochs to 5 the loss is:StepTraining Loss5000.32020010000.24680015000.23060020000.17120025000.16080030000.15240035000.10240040000.08570045000.09860050000.06640055000.05080060000.04540065000.03350070000.03050075000.030600But when I set  num_train_epochs to 1 the loss is:StepTraining Loss5000.33890010000.24290015000.212500If I am running again with num_train_epochs to 1 I am getting the exactly same loss:StepTraining Loss5000.33890010000.24290015000.212500So my questions is: Why different num_train_epochs gives different loss? How to make it consistent? e.g., how to make that on num_train_epochs = 1 I will receive the same 3 losses as in num_train_epochs = 5
2023-08-17T19:09:53Z
What learning rate schedule are you using? Some LR schedules (eg cosine) decay the learning rate based on the total number of training steps. Eg it decays faster if you train for fewer epochs.