Fix code scanning alert - Potentially uninitialized local variable
Please fix this bug: `affine_params` would be missing when `self.affine_model not in ["wavenet", "simple_conv"]`

<!-- Warning: The suggested title contains the alert rule name. This can expose security information. -->

Tracking issue for:
- [x] https://github.com/NVIDIA/NeMo/security/code-scanning/774
