Optimizing Learning Rate Parameters in Model Fine-tuning
This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days. What script are you running, could you share the command? Closing since no follow-up. Hi @ericharper  ,

Thank you for your time and help.

I made a few changes to the base NeMo toolkit script. The bug I was facing was due to an script not reflecting in my container and some hyper-parameter values. I fixed the container, updated the scripts, and adjusted a few hyper-parameters. It's working now.

Thanks again! 😊
I've been working on fine-tuning the model within a specific domain and tweaking various hyperparameters, including learning rates, via a YAML file during training. However, upon completing the training process, I noticed that the toolkit defaults to using the learning rate from the last pre-trained model instead of considering the parameters specified in the YAML file. Is there a way to ensure that it prioritizes the learning rate parameter I've passed from the Yaml file?

Additionally, post fine-tuning, I've observed that the model fails to learn effectively. Despite a significant increase in training duration (from around 1200 hours in the baseline to approximately 4000 hours in the fine-tuned model), and the use of noise augmentation during fine-tuning compared to the clean data used in the baseline, the results remain unchanged. Would it be reasonable to conclude that fine-tuning isn't yielding significant improvements, or it's more because of the learning rate default value which is considered from the last pre-trained model?