What is the meaning of "steps" parameters?
I’m not sure I understand the meaning of “step” inrun_speech_recognition_seq2seq(Trainer)what is the meaning of step ? what is the relation to epochs ?whensave_total_limit= 5,does it mean that always thebest (metric)5 steps are saved ? orthe last 5 steps are saved (and they may not be the with the best metric) ?
2023-09-26T10:39:19Z
what is the meaning of step ? what is the relation to epochs ?A “step” (also called “training step” or “optimization step”) is a single forward pass + backward pass through the model. The model takes in a batch of examples, computes the loss and gradients, and then updates the model’s parameters during the backward pass. That all happens during a single training step.The relationship with an epoch is that an epoch is one pass through the full training set (the model has seen every training example 1 time). Suppose you have 8000 training examples and a batch size of 8. One epoch will consist of 1000 steps.whensave_total_limit= 5,does it mean that always thebest (metric)5 steps are saved ? orthe last 5 steps are saved (and they may not be the with the best metric) ?Quick comment on terminology - the word you’re looking for here is “checkpoints” not stepsTo answer the question, there’s documentation for this ishere. They say:Whenload_best_model_at_endis enabled, the “best” checkpoint according tometric_for_best_modelwill always be retained in addition to the most recent ones. For example, forsave_total_limit=5andload_best_model_at_end, the four last checkpoints will always be retained alongside the best model.So you wanna setload_best_model_at_end=Truein theTrainingArgumentsand HF will keep the best checkpoint along with the most recent ones.