cosine annealing linear warmup lr=0 on first step
I'm not sure we want to modify that lr schedule as it's been used as-is for so long. Can you use your own implementation? NeMo supports users to use their own lr schedules.

https://docs.nvidia.com/nemo-framework/user-guide/latest/nemotoolkit/core/core.html?highlight=learning%2520rate#learning-rate-schedulers

**Describe the bug**

Please see https://github.com/NVIDIA/NeMo/pull/10530

**Steps/Code to reproduce bug**

Please see https://github.com/NVIDIA/NeMo/pull/10530

**Expected behavior**

LR != 0 on a first step

**Environment overview (please complete the following information)**

 - Environment location: [Bare-metal, Docker, Cloud(specify cloud provider - AWS, Azure, GCP, Collab)]: Any
 - Method of NeMo install: [pip install or from source]. Please specify exact commands you used to install: Any
 - If method of install is [Docker], provide `docker pull` & `docker run` commands used: N/A

**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version: Any
- PyTorch version: Any
- Python version: Any

**Additional context**

Add any other context about the problem here.
Example: GPU model: Any
