Hyperparameters for lr_scheduler_type in Trainer Arguments
My understanding is that I can pass the following string values to use the corresponding learning rate schedulers in Training Arguments:“linear” → transformers.get_linear_schedule_with_warmup“cosine” → transformers.get_cosine_schedule_with_warmup“cosine_with_restarts” -->transformers.get_cosine_with_hard_restarts_schedule_with_warmup“polynomial” → transformers.get_polynomial_decay_schedule_with_warmup“constant” → transformers.get_constant_schedule“constant_with_warmup” → transformers.get_constant_schedule_with_warmup“inverse_sqrt” → transformers.get_inverse_sqrt_scheduleHowever, some of the schedulers have specific hyperparameters. For example, transformers.get_cosine_schedule_with_warmup has num_cycels as a parameter. How can we change the default values?
2024-01-29T16:22:32Z
Hi, there is a another argument calledlr_scheduler_kwargs, you can change the hyperparameters in this with dict.
2024-03-05T00:35:35Z
which version have the argument calledlr_scheduler_kwargs. i used transformers   4.28.0